Multi-modal academic image tampering detection method and system based on frequency domain gating attention and weakly supervised iteration

Abstract

This invention relates to the fields of digital image forensics and artificial intelligence, and particularly to a multimodal academic image tampering detection method and system based on frequency-domain gated attention and weakly supervised iteration. The method includes: acquiring an image of the academic document to be detected and its corresponding text; performing SRM filtering and DCT transformation on the image to construct a frequency-domain noise residual map; extracting RGB spatial features, frequency-domain noise features, and text semantic features from the image; inputting the above features into a frequency-domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA, using the frequency-domain noise features to generate gated mask modulation attention weights, and outputting multimodal fusion features; calculating logical consistency scores and physical tampering scores based on the fusion features for judgment; generating pseudo-labels based on high-confidence detection results to drive weakly supervised iterative updates of the model. This application solves the problems of insufficient utilization of frequency-domain features, inability to detect logical fallacies in the image and text, and difficulty in continuous model evolution.

Description

Technical Field

[0001] This invention relates to the fields of digital image forensics and artificial intelligence, and in particular to a multimodal academic image tampering detection method and system based on frequency domain gated attention and weakly supervised iteration. Background Technology

[0002] In recent years, the number of academic papers has surged, and image manipulation, particularly in the biomedical field, has become a major area of ​​academic misconduct. Common manipulation methods have evolved from simple copy-paste to sophisticated stitching, content erasure, and deepfakes using image processing software, seriously threatening research integrity.

[0003] Currently, technologies for detecting tampering with academic images mainly face the following technical bottlenecks: Low utilization of frequency domain features: Existing deep learning methods mostly focus on visual feature extraction in the RGB domain, with insufficient exploration of the underlying physical features of images. For stitched images that have undergone refined post-processing (such as edge feathering), their visual traces in the RGB domain are weak. Although some methods have introduced frequency domain analysis, they usually only perform simple channel concatenation of frequency domain features with RGB features. This "unguided" fusion method makes it difficult for the model to focus on the frequency domain noise discontinuities introduced by tampering operations, making the model susceptible to interference from high-frequency texture content in the image, resulting in a high false negative rate.

[0004] Lack of cross-modal logical consistency detection: Existing technologies are mostly unimodal models, focusing only on pixel-level tampering of the image itself. However, academic fraud often manifests as logical fallacies such as "inconsistency between image and text" or "using one image for multiple purposes" (e.g., using the same image to represent different experimental conditions). Traditional models cannot understand textual semantics and therefore cannot detect this type of non-pixel modification-based logical fraud.

[0005] Insufficient generalization and evolutionary capabilities of models: Academic fraud methods are constantly evolving, while obtaining high-quality, diverse pixel-level labeled data is extremely costly. Once deployed, the performance of supervised learning-based detection models rapidly degrades as they fail to adapt to emerging fraud patterns. How to enable models to continuously learn and evolve in massive data streams lacking strong supervision signals is a major challenge currently facing the technology.

[0006] Therefore, there is an urgent need for a multimodal academic image tampering detection method and system based on frequency domain gated attention and weakly supervised iteration to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art, this invention provides a multimodal academic image tampering detection method and system based on frequency domain gated attention and weakly supervised iteration, in order to solve the problems existing in the background art.

[0008] This invention provides the following technical solution: a multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration, comprising the following steps: S100: Acquire the image data and corresponding title text data of the academic document to be detected; S200: Perform spatial rich model (SRM) filtering and local discrete cosine transform (DCT) processing on the image data to construct a frequency domain noise residual map; S300: Construct a multi-stream feature extraction network, including an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder, to extract spatial visual features, frequency domain noise features, and text semantic features of the image, respectively. S400: Input the spatial visual features, the frequency noise features, and the text semantic features into the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA, use the frequency noise features to generate a spatial gating mask, modulate the cross-modal attention weights between the spatial visual features and the text semantic features, and output multimodal fusion features; S500: Based on the multimodal fusion features, calculate the logical consistency score and the physical tampering score. If the score exceeds the preset threshold, it is determined to be a tampered image. Based on the detection results, high-confidence samples are selected to generate pseudo-labels, driving the model to perform weakly supervised self-iterative updates.

[0009] Furthermore, the specific construction process of the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA in step S400 includes: Step A: Frequency domain gated signal generation; using convolutional layers to analyze the frequency domain noise features. Dimensionality reduction and nonlinear activation are performed to generate spatially sensitive gated masks. The calculation formula is: ; in, It is the Sigmoid activation function. This represents the convolution operation. and These are learnable parameters; Step B: Gated modulation attention calculation; mapping the text semantic features to a query vector $Q$, and mapping the spatial visual features to a key vector. Sum value vector Introducing the gate mask Modulate the attention score matrix: .; in, It represents the Hadamah accumulation. This is the gain coefficient. This is the bias coefficient; Step C: Dual-granularity feature fusion; output a fine-grained feature map containing pixel-level positioning information, and perform global average pooling on the feature map, matching it with the text entity vector to obtain entity-level coarse-grained features.

[0010] Furthermore, the step S200, which involves performing spatial rich model (SRM) filtering and local discrete cosine transform (DCT) processing on the image data, includes: The image is convolved using an SRM filter bank that includes first-order, second-order, and third-order differences to extract high-frequency residual maps. ; The image is divided into 8×8 non-overlapping blocks. A DCT transform is performed on each block, and the histogram statistical characteristics of the AC coefficients are calculated. ; The high-frequency residual map With the histogram statistical features Cascaded along the channel dimension to form an enhanced frequency domain input tensor.

[0011] Furthermore, in step S500, the step of calculating the logical consistency score and the physical tampering score employs a multi-task joint loss function, the formula of which is as follows: ; in, To tamper with the classification of losses, For frequency domain reconstruction loss, The logically mutually exclusive comparison loss; the logically mutually exclusive comparison loss Defined as: ; in, Indicates the visual similarity between images. Indicates the semantic distance between texts. For similarity threshold, This represents the semantic boundary.

[0012] Furthermore, the step S500, which involves selecting high-confidence samples based on the detection results to generate pseudo-labels and drive the model to perform weakly supervised self-iterative updates, includes: Construct a dynamic experience replay pool to store historical high-confidence samples; In unlabeled data streams, the logically mutually exclusive comparison loss is utilized. Filter out logically conflicting samples and mark them with the pseudo-label "logical tampering"; The physical tampering score is used to filter out samples with discontinuous noise distribution, which are then marked as pseudo-labels of "splicing tampering". A momentum update strategy is adopted, which uses the pseudo-label data to mix with the data in the experience replay pool to fine-tune the model parameters online.

[0013] A multimodal academic image tampering detection system based on frequency-domain gated attention and weakly supervised iteration includes: The data acquisition and preprocessing module is used to acquire academic literature data and perform RGB image normalization, SRM filtering and text cleaning in parallel. The multi-stream feature extraction module includes an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder; The FG-DGCMA inference engine uses frequency domain noise features to generate a spatial gate mask, and uses the gate mask to modulate the cross-modal attention weights between RGB spatial domain visual features and text semantic features, outputting multimodal fusion features to achieve deep alignment of image and text features and anomaly detection. A weakly supervised self-learning controller is used to automatically generate pseudo-labels based on anomaly scores, maintain an experience replay pool, and trigger backpropagation updates of the model. The visual evidence collection module is used to generate a detection report that overlays a frequency domain noise heatmap and a logical conflict box.

[0014] Furthermore, the RGB visual encoder uses a Swing Transformer network; the frequency domain noise encoder uses an EfficientNet network.

[0015] Furthermore, the weakly supervised self-learning controller also includes an uncertainty sampling unit, which is used to perform Monte Carlo Dropout inference on samples whose prediction confidence is within a preset fuzzy range, calculate the prediction variance, and add the sample to the manual review queue if the variance is higher than a threshold.

[0016] An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of any of the methods described above.

[0017] A computer-readable storage medium, characterized in that the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of any of the methods described above.

[0018] The technical effects and advantages of this invention are as follows: Improve the efficiency of frequency domain feature utilization: Construct a frequency domain noise residual map through SRM filtering and DCT transformation, and use the frequency domain gating mask in the FG-DGCMA module to modulate the attention weights, so that the model can focus on the noise discontinuity region introduced by the tampering operation, reduce the interference of image content on detection, and improve the detection capability of fine splicing tampering.

[0019] Achieving image-text logical consistency detection: By using a dual-granularity cross-modal attention mechanism, entity-level image-text matching is introduced on the basis of pixel-level localization. Combined with a logically mutually exclusive contrast loss function, it can identify image-text logical fallacies that are "visually similar but semantically mutually exclusive", expanding the detection range to non-pixel-modified image reuse forgery.

[0020] Supports online self-iteration of the model: By generating pseudo-labels through high-confidence samples, constructing a dynamic experience replay pool, and adopting a momentum update strategy, the model can continuously perform weakly supervised learning in unlabeled data streams, adapt to new fraud methods, and reduce dependence on manually labeled data. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration provided in this application embodiment; Figure 2 A detailed architectural diagram of the multi-stream feature extraction network and FG-DGCMA module provided in the embodiments of this application; Figure 3 The computational logic diagram of the frequency domain saliency-gated attention mechanism provided in the embodiments of this application is shown.

[0022] Figure 4 The structural block diagram of the multimodal academic image tampering detection system based on frequency domain gated attention and weakly supervised iteration provided in the embodiments of this application is shown. Detailed Implementation

[0023] The technical solution of the present invention will be described in detail below. This embodiment is only used to explain the present invention and is not intended to limit the scope of protection of the present invention.

[0024] This invention provides the following technical solution: a multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration, comprising the following steps: S100: Acquire the image data and corresponding title text data of the academic document to be detected; S200: Perform spatial rich model SRM filtering and local discrete cosine transform (DCT) processing on image data to construct a frequency domain noise residual map; S300: Construct a multi-stream feature extraction network, including an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder, to extract spatial visual features, frequency domain noise features, and text semantic features of the image, respectively. S400: Inputs spatial visual features, frequency noise features and text semantic features into the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA, uses frequency domain noise features to generate a spatial gating mask, modulates the cross-modal attention weights between spatial visual features and text semantic features, and outputs multimodal fusion features. S500: Based on multimodal fusion features, it calculates logical consistency score and physical tampering score. If the score exceeds the preset threshold, it is determined to be a tampered image. Based on the detection results, it selects high-confidence samples to generate pseudo-labels and drives the model to perform weakly supervised self-iterative updates.

[0025] Specifically, the construction process of the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA in step S400 includes: Step A: Frequency domain gated signal generation; using convolutional layers to analyze frequency domain noise features. Dimensionality reduction and nonlinear activation are performed to generate spatially sensitive gated masks. The calculation formula is: ; in, It is the Sigmoid activation function. This represents the convolution operation. and These are learnable parameters; Step B: Gated modulation attention calculation; mapping text semantic features to query vector $Q$, and spatial visual features to key vectors. Sum value vector Introducing gating masks Modulate the attention score matrix: .; in, It represents the Hadamah accumulation. This is the gain coefficient. This is the bias coefficient; Step C: Dual-granularity feature fusion; output a fine-grained feature map containing pixel-level positioning information, and perform global average pooling on the feature map, matching it with the text entity vector to obtain entity-level coarse-grained features.

[0026] Specifically, step S200, which involves performing spatial rich model (SRM) filtering and local discrete cosine transform (DCT) processing on the image data, includes: The image is convolved using an SRM filter bank that includes first-order, second-order, and third-order differences to extract high-frequency residual maps. ; The image is divided into 8×8 non-overlapping blocks. A DCT transform is performed on each block, and the histogram statistical characteristics of the AC coefficients are calculated. ; High-frequency residual map Histogram statistical characteristics Cascaded along the channel dimension to form an enhanced frequency domain input tensor.

[0027] Specifically, in step S500, the step of calculating the logical consistency score and the physical tampering score uses a multi-task joint loss function, the formula of which is as follows: ; in, To tamper with the classification of losses, For frequency domain reconstruction loss, For logically mutually exclusive comparison loss; logically mutually exclusive comparison loss Defined as: ; in, Indicates the visual similarity between images. Indicates the semantic distance between texts. For similarity threshold, This represents the semantic boundary.

[0028] Specifically, step S500, which involves selecting high-confidence samples based on the detection results to generate pseudo-labels and drive the model to perform weakly supervised self-iterative updates, includes: Construct a dynamic experience replay pool to store historical high-confidence samples; In unlabeled data streams, logical mutual exclusion is used to compare loss. Filter out logically conflicting samples and mark them with the pseudo-label "logical tampering"; Samples with discontinuous noise distribution were screened out by physically altering the scores and marked as pseudo-labels of "splicing and tampering"; A momentum update strategy is adopted, which uses pseudo-label data and data from the experience replay pool to fine-tune the model parameters online.

[0029] A multimodal academic image tampering detection system based on frequency-domain gated attention and weakly supervised iteration includes: The data acquisition and preprocessing module is used to acquire academic literature data and perform RGB image normalization, SRM filtering and text cleaning in parallel. The multi-stream feature extraction module includes an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder; The FG-DGCMA inference engine uses frequency domain noise features to generate spatial gating masks, and uses the gating masks to modulate the cross-modal attention weights between RGB spatial domain visual features and text semantic features, outputting multimodal fusion features to achieve deep alignment of image and text features and anomaly detection. A weakly supervised self-learning controller is used to automatically generate pseudo-labels based on anomaly scores, maintain an experience replay pool, and trigger backpropagation updates of the model. The visual evidence collection module is used to generate a detection report that overlays a frequency domain noise heatmap and a logical conflict box.

[0030] Specifically, the RGB visual encoder uses the Swing Transformer network; the frequency domain noise encoder uses the EfficientNet network.

[0031] Specifically, the weakly supervised self-learning controller also includes an uncertainty sampling unit, which is used to perform Monte Carlo Dropout inference on samples whose prediction confidence is within a preset fuzzy range, calculate the prediction variance, and add the sample to the manual review queue if the variance is higher than the threshold.

[0032] An electronic device, characterized in that it comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0033] A computer-readable storage medium is characterized in that it stores instructions that, when executed on a computer, cause the computer to perform the steps of any of the methods described above.

[0034] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0035] Example 1: Detection Method Flow like Figure 1 As shown in the figure, this embodiment provides a multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration, including the following steps S100 to S500.

[0036] S100: Multimodal data acquisition and frequency domain preprocessing Obtain the PDF file of the academic document to be detected, parse and extract image data I and its title text data T. Perform a dual frequency domain transform on image data I: The first step is to perform SRM filtering. The image is convolved using 30 fixed-weight SRM filter kernels (such as 5×5 difference kernels), outputting a 30-channel noise residual map. The SRM filter bank effectively suppresses image content and highlights the noise residuals in the image.

[0037] The second step involves DCT transformation. The image is converted to the YCbCr color space, the Y channel is extracted, and the image is divided into 8×8 non-overlapping blocks. DCT transformation is performed on each block, and the quantization coefficient histogram of each block is calculated. The DCT histogram features are sensitive to JPEG compression artifacts and can identify spliced ​​regions that have undergone double compression or compression with different quality factors.

[0038] The SRM residual map and DCT feature map are stacked along the channel dimension to generate a frequency domain input tensor. .

[0039] S200: Parallel Extraction of Multi-Stream Features Construct a three-stream neural network architecture and perform feature extraction for each stream: The first stream is an RGB spatial stream, used to extract spatial visual features. The original image is input into the SwingTransformer network, which utilizes its hierarchical Shifted Window Attention mechanism to extract spatial features containing semantic information.

[0040] The second stream is the frequency domain noise stream, used to extract frequency domain noise features. Input frequency domain tensor Input the EfficientNet-B4 network; this branch learns the noise distribution features of the image and outputs frequency domain features. .

[0041] The third stream is the text semantic stream, used to extract text semantic features. Input the text T into the BioBERT model to extract the global feature vector of the text and the entity embedding vector of each word.

[0042] S300: Frequency Domain Gated Attention Fusion This step utilizes the FG-DGCMA module to implement frequency-domain guided cross-modal attention fusion. A gating mask generated using F_freq is used to modulate the attention calculation process.

[0043] The specific calculation process is as follows: using convolutional layers to analyze frequency domain noise features Dimensionality reduction and nonlinear activation are performed to generate spatially sensitive gated masks. The calculation formula is: The mask The value indicates the degree of noise anomaly in the area. A value close to 1 indicates a high probability of noise anomaly, while a value close to 0 indicates good noise continuity.

[0044] Calculate the attention score after gating and modulation: Map text semantic features to query vector Q, and spatial visual features to key vector K and value vector V, and introduce a gating mask. The attention score matrix is ​​modulated and calculated using the following formula: ; in, For element-wise multiplication, This is the gain coefficient. Here is the bias coefficient. The effect of this formula is: the multiplication enhancement term (1 + λ· This amplifies the attention weights in noisy, abnormal regions; the additive bias term β· This ensures that areas with abnormal noise receive basic attention allocation.

[0045] Based on the modulated attention weights, output multimodal fusion features. This module employs a dual-granularity output mechanism: on one hand, it outputs a fine-grained feature map containing pixel-level positioning information; on the other hand, it performs global average pooling on this feature map and matches it with text entity vectors to obtain entity-level coarse-grained features.

[0046] S400: Anomaly Scoring Calculation and Judgment Based on the fusion features, scores are calculated across two dimensions: The first dimension is the logical mutual exclusion score. The cosine similarity S_vis of the visual features of the image pair (I_i, I_j) is calculated, along with the semantic distance D_text of the text features. If S_vis > 0.95 and D_text > 0.5, it is determined to be logical tampering.

[0047] The second dimension is the physical tampering score. The fused feature F_fused is input into the fully convolutional decoder, which outputs a pixel-level heatmap. The area and intensity of the highlighted areas in the heatmap are calculated, and if the score exceeds a preset threshold, it is determined to be a splicing tampering.

[0048] During the model training phase, a multi-task joint loss function is used for optimization, and its formula is as follows: ; in, To tamper with the classification of losses, For frequency domain reconstruction loss, This is the logical mutual exclusion comparison loss. The logical mutual exclusion comparison loss is defined as follows: ; S500: Weakly Supervised Self-Iteration After the model is deployed, online inference is performed on the unlabeled data.

[0049] A dynamic experience replay pool is constructed to store historical high-confidence samples. Logically mutually exclusive contrastive loss is used in the unlabeled data stream. Samples with logical conflicts were selected and marked with the pseudo-label "logical tampering"; samples with discontinuous noise distribution were selected using physical tampering scores and marked with the pseudo-label "splicing tampering".

[0050] Select tampered samples and real samples with confidence scores higher than a preset threshold (e.g., 0.98), and store these samples and their pseudo-labels in a dynamic experience replay pool.

[0051] A momentum update strategy is adopted to sample data from the replay pool and use pseudo-label data to mix with historical data in the experience replay pool to fine-tune the model through backpropagation.

[0052] Example 2: Details of the FG-DGCMA Module like Figure 2 and Figure 3 As shown, the FG-DGCMA module includes a dual-granularity output mechanism.

[0053] At the pixel-level granularity, it is used to locate the tampered region. The Attention Map output by the module corresponds to the spatial resolution of the image, and an upsampling operation can generate a heatmap for locating the tampered region.

[0054] At the entity level, this is used for logical judgment. The module aggregates image features into Visual Tokens and matches them with Entity Tokens in the text. When a Visual Token in the image and a semantically exclusive Entity Token in the text are too close in the feature space, a logical alarm is triggered.

[0055] Example 3: System Architecture like Figure 4 As shown in the figure, this embodiment provides a multimodal academic image tampering detection system based on frequency domain gated attention and weakly supervised iteration, which includes the following modules: The data acquisition and preprocessing module is used to acquire academic literature data and perform RGB image normalization, SRM filtering, and text cleaning in parallel.

[0056] The multi-stream feature extraction module includes an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder. The RGB visual encoder uses a Swing Transformer network, while the frequency domain noise encoder uses an EfficientNet network.

[0057] The FG-DGCMA inference engine is used to achieve deep alignment of image and text features and anomaly detection guided by frequency domain noise features. This engine uses frequency domain noise features to generate spatial gating masks, modulates the cross-attention weights of visual and text features, and outputs multimodal fusion features.

[0058] A weakly supervised self-learning controller is used to automatically generate pseudo-labels based on anomaly scores, maintain an experience replay pool, and trigger backpropagation updates of the model. This controller also includes an uncertainty sampling unit, used to perform Monte Carlo Dropout inference on samples with prediction confidence within a preset fuzzy range, calculate prediction variance, and add the sample to a manual review queue if the variance exceeds a threshold.

[0059] The visual evidence collection module is used to generate a detection report that overlays a frequency domain noise heatmap and a logical conflict box.

[0060] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration, characterized in that, Includes the following steps: S100: Acquire the image data and corresponding title text data of the academic document to be detected; S200: Perform spatial rich model (SRM) filtering and local discrete cosine transform (DCT) processing on the image data to construct a frequency domain noise residual map; S300: Construct a multi-stream feature extraction network, including an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder, to extract spatial visual features, frequency domain noise features, and text semantic features of the image, respectively. S400: Input the spatial visual features, the frequency noise features, and the text semantic features into the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA, use the frequency noise features to generate a spatial gating mask, modulate the cross-modal attention weights between the spatial visual features and the text semantic features, and output multimodal fusion features; S5 00: Based on the multimodal fusion features, calculate the logical consistency score and the physical tampering score. If the score exceeds the preset threshold, it is determined to be a tampered image. Based on the detection results, high-confidence samples are selected to generate pseudo-labels, driving the model to perform weakly supervised self-iterative updates.

2. The multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration according to claim 1, characterized in that, The specific construction process of the frequency domain saliency-gated dual-granularity cross-modal attention module FG-DGCMA in step S400 includes: Step A: Frequency domain gated signal generation; using convolutional layers to analyze the frequency domain noise features. Dimensionality reduction and nonlinear activation are performed to generate spatially sensitive gated masks. The calculation formula is: ； in, It is the Sigmoid activation function. This represents the convolution operation. and These are learnable parameters; Step B: Gated modulation attention calculation; mapping the text semantic features to a query vector $Q$, and mapping the spatial visual features to a key vector. Sum value vector Introducing the gate mask Modulate the attention score matrix: .； in, It represents the Hadamah accumulation. This is the gain coefficient. This is the bias coefficient; Step C: Dual-granularity feature fusion; output a fine-grained feature map containing pixel-level positioning information, and perform global average pooling on the feature map, matching it with the text entity vector to obtain entity-level coarse-grained features.

3. The multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration according to claim 1, characterized in that, The step S200, which involves performing spatial rich model (SRM) filtering and local discrete cosine transform (DCT) processing on the image data, includes: The image is convolved using an SRM filter bank that includes first-order, second-order, and third-order differences to extract high-frequency residual maps. ; The image is divided into 8×8 non-overlapping blocks. A DCT transform is performed on each block, and the histogram statistical characteristics of the AC coefficients are calculated. ; The high-frequency residual map With the histogram statistical features Cascaded along the channel dimension to form an enhanced frequency domain input tensor.

4. The multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration according to claim 1, characterized in that, In step S500, the calculation of the logical consistency score and the physical tampering score uses a multi-task joint loss function, the formula of which is as follows: ； in, To tamper with the classification of losses, For frequency domain reconstruction loss, The logically mutually exclusive comparison loss; the logically mutually exclusive comparison loss Defined as: ； in, Indicates the visual similarity between images. Indicates the semantic distance between texts. For similarity threshold, This represents the semantic boundary.

5. The multimodal academic image tampering detection method based on frequency domain gated attention and weakly supervised iteration according to claim 1 or 4, characterized in that, The step S500, which involves selecting high-confidence samples based on the detection results to generate pseudo-labels and driving the model to perform weakly supervised self-iterative updates, includes: Construct a dynamic experience replay pool to store historical high-confidence samples; In unlabeled data streams, the logically mutually exclusive comparison loss is utilized. Filter out logically conflicting samples and mark them with the pseudo-label "logical tampering"; The physical tampering score is used to filter out samples with discontinuous noise distribution, which are then marked as pseudo-labels of "splicing tampering"; A momentum update strategy is adopted, which uses the pseudo-label data to mix with the data in the experience replay pool to fine-tune the model parameters online.

6. A multimodal academic image tampering detection system based on frequency-domain gated attention and weakly supervised iteration, characterized in that, include: The data acquisition and preprocessing module is used to acquire academic literature data and perform RGB image normalization, SRM filtering and text cleaning in parallel. The multi-stream feature extraction module includes an RGB visual encoder, a frequency domain noise encoder, and a text semantic encoder; The FG-DGCMA inference engine uses frequency domain noise features to generate a spatial gate mask, and uses the gate mask to modulate the cross-modal attention weights between RGB spatial domain visual features and text semantic features, outputting multimodal fusion features to achieve deep alignment of image and text features and anomaly detection. A weakly supervised self-learning controller is used to automatically generate pseudo-labels based on anomaly scores, maintain an experience replay pool, and trigger backpropagation updates of the model. The visual evidence collection module is used to generate a detection report that overlays a frequency domain noise heatmap and a logical conflict box.

7. The system according to claim 6, characterized in that, The RGB visual encoder uses a SwinTransformer network; the frequency domain noise encoder uses an EfficientNet network.

8. The system according to claim 6, characterized in that, The weakly supervised self-learning controller also includes an uncertainty sampling unit, which is used to perform Monte Carlo Dropout inference on samples whose prediction confidence is in a preset fuzzy range, calculate the prediction variance, and add the sample to the manual review queue if the variance is higher than the threshold.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the method as described in any one of claims 1 to 5.

Images