General image forgery detection method and system based on two-path difference checking

Abstract

The application discloses a kind of general image forgery detection method and system based on two-way difference check, belong to image processing technical field.Method includes obtaining detection image and semantic anchor point text, respectively using pre-trained visual language model to image and semantic anchor point text is encoded processing, obtain image global feature and anchor point text feature, with two similarity scores as virtual loss score to perform back propagation, extract key level gradient tensor and output signal dimension forgery probability by gradient meta adapter;Through image text description generation and reverse image generation constructs check closed loop, measure multi-feature consistency in cross-modal shared feature space, obtain semantic dimension forgery probability;Based on image global feature generation dynamic weight, weighted fusion two-way probability output final detection result;Method realizes the comprehensive detection of different types of synthetic fake image, can maintain excellent detection precision and generalization performance in the test scene of multi-source generation model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a general image forgery detection method and system based on dual-channel difference verification. Background Technology

[0002] With the rapid iteration of Artificial Intelligence Generated Content (AIGC) technology, image generation technologies, represented by generative adversarial networks and diffusion models, have made breakthrough progress, and the visual realism of synthesized images has reached a level that is difficult to distinguish with the naked eye. While AIGC technology is driving the innovative development of the digital content industry, its risk of abuse is also becoming increasingly prominent. Forged images are maliciously used in scenarios such as the spread of fake news, online fraud, identity forgery, and manipulation of public opinion, threatening cyberspace security and public interests. Therefore, developing effective image forgery detection technology as a core line of defense against the risk of AIGC abuse has become an urgent task in the field of cyberspace security.

[0003] In related technologies, image forgery detection methods typically employ two forensic approaches: active and passive. In the active approach, an externally added signal (i.e., a watermark) is embedded into the source image without producing visual artifacts. To determine if an image has been tampered with, a watermark extraction process is performed on the target image to restore the watermark; the extracted watermark image can then be used to detect tampered areas in the target image. The passive approach to image forgery detection utilizes statistical information from the source image, specifically the high consistency between different images, to detect forged areas. However, existing forgery detection methods suffer from at least the following problems: (1) Over-reliance on image underlying microscopic signal artifacts makes it prone to failure in the face of high-fidelity generation models, and it is difficult to deal with pixel-perfect forged images; (2) Lacking semantic logic verification capabilities, it is unable to identify forged samples whose content violates physical common sense and event logic. Summary of the Invention

[0004] This application aims to propose a general image forgery detection method and system based on dual-path difference verification, in order to at least solve the technical problems of image forgery detection methods in related technologies that rely too much on the microscopic signal artifacts at the bottom layer of the image, are prone to failure in the face of high-fidelity generation models, are difficult to deal with pixel-perfect forged images, and lack semantic logic verification capabilities, making it impossible to identify forged samples whose content violates physical common sense and event logic.

[0005] In a first aspect, this application provides a general image forgery detection method based on dual-channel difference verification, comprising: The detection image and semantic anchor text are acquired, and a pre-trained visual language model is used to encode the detection image and the semantic anchor text respectively to obtain the global features of the image and the features of the anchor text. Based on the global features of the image and the anchor text features, a similarity score is determined, and the similarity score is used as a virtual loss function to perform backpropagation processing to extract the gradient tensors of preset key levels within the model. The gradient tensor is input into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity discrimination, and the output signal dimension forgery probability is obtained. The detected image is subjected to textual description of visual content to obtain a target text description, and the target text description is subjected to image generation processing to obtain a reference image; Based on a unified cross-modal shared feature space, visual features of the target image, text features of the target text description, and reference visual features of the reference image are extracted respectively. Semantic authenticity is determined based on visual features, text features, and reference visual features to determine the semantic dimension forgery probability. Based on the dynamic weight coefficients calculated from the global features of the image, a weighted fusion calculation is performed on the forgery probability of the signal dimension and the forgery probability of the semantic dimension to obtain the image forgery detection result.

[0006] In some embodiments, the step of determining a similarity score based on the global features of the image and the anchor text features, using the similarity score as a virtual loss function to perform backpropagation processing, and extracting the gradient tensors of preset key levels within the model includes: Calculate the cosine similarity between the global features of the image and the anchor text features in the cross-modal shared feature space, and use it as a similarity score representing the true attributes of the image; The similarity score is used as a virtual loss function to perform virtual backpropagation, wherein the backpropagation process does not update any weight parameters of the visual language model, but is only used to trigger the internal gradient response of the visual language model for the real semantic anchor point. The multi-layer coding blocks of the image encoder in the visual language model are selected as preset key layers, and the gradient tensors flowing through each key layer are extracted respectively. The gradient tensors contain gradient magnitude, gradient direction and gradient spatial distribution information.

[0007] In some embodiments, the step of inputting the gradient tensor into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity determination, and outputting the signal dimension forgery probability, includes: The gradient tensors of each key level are flattened and linearly projected to obtain the gradient feature sequences of the corresponding dimensions. Learnable positional codes are added to the gradient feature sequences to distinguish the model layers corresponding to the gradients; The gradient feature sequence with added position encoding is input into a Transformer encoder with a preset number of layers for global feature aggregation to obtain aggregated gradient features; The aggregated gradient features are input into a linear classifier, which outputs the signal dimension spoofing probability.

[0008] In some embodiments, the step of generating a textual description of the visual content of the detected image to obtain a target text description, and performing image generation processing on the target text description to obtain a reference image, includes: Semantic parsing and natural language conversion are performed on the visual content of the detected images to generate target text descriptions covering the main object, attribute features, spatial relationships and scene information; The target text description is used as a semantic constraint to perform image semantic reconstruction processing, generating a reference image that semantically matches the target text description.

[0009] In some embodiments, the step of determining the semantic dimension forgery probability by performing semantic authenticity judgment based on visual features, text features, and reference visual features includes: Calculate the cosine similarity between the visual features and the text features, and use it as the image-text consistency score; Calculate the cosine similarity between the visual feature and the reference visual feature, as the graph. Figure 1 Consistency score; The image-text consistency score is compared with the image... Figure 1 Consistency scores are concatenated into a two-dimensional semantic feature vector, which is then input into a multilayer perceptron for authenticity determination, and the output is the semantic dimension forgery probability.

[0010] In some embodiments, the step of performing a weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability based on the dynamic weight coefficients calculated from the global features of the image to obtain the image forgery detection result includes: The global features of the image are input into a preset gated network. After feature mapping is performed through a linear transformation layer, the output value is constrained to a normalized numerical range through a nonlinear activation function to obtain dynamic weight coefficients in single scalar form. The dynamic weight coefficient is used as the weighting coefficient for the signal dimension forgery probability, and the complementary value corresponding to the dynamic weight coefficient is used as the weighting coefficient for the semantic dimension forgery probability. A weighted summation operation is performed on the signal dimension forgery probability and the semantic dimension forgery probability to output the image forgery detection result.

[0011] In some embodiments, the trainable parameters of the preset gating network, the gradient aggregation fitting parameters corresponding to signal dimension detection, and the multilayer perceptron parameters corresponding to semantic dimension detection are updated through end-to-end backpropagation using a unified classification loss function during the model training phase.

[0012] Compared with the prior art, the technical solution provided in the first aspect of this application includes at least the following beneficial effects or advantages: The detection method provided in this application constructs a parallel detection architecture encompassing both signal and semantic dimensions. On one hand, it uses the similarity score between global image features and anchor text features as a virtual loss to perform backpropagation, extracting gradient tensors from key model levels and a gradient meta-adaptor to complete multi-level feature aggregation and discrimination. This approach leverages the pre-trained visual language model's inherent representation of the statistical regularities of natural images to capture abnormal patterns at the image's underlying signal level, without relying on displayed pixel artifact features. It maintains stable detection and discrimination capabilities even when facing high-fidelity generated forged images. On the other hand, it constructs a verification closure through textual description of image content and reverse image generation. The ring performs consistency measurement of multi-source features within a unified cross-modal shared feature space, enabling it to determine the logical rationality of image content from a macro-semantic perspective. It covers forgery scenarios where there are no obvious flaws at the pixel level, but the content does not conform to physical common sense and event logic. Simultaneously, it generates dynamic weight coefficients based on global image features and adaptively weights and fuses the two discrimination results. It can match the weight ratio of different detection dimensions according to the characteristics of the input image, integrating two types of discrimination information: micro-signal anomalies and macro-semantic deviations. This achieves comprehensive detection of different types of synthetic forged images and maintains excellent detection accuracy and generalization performance in the test scenarios of multi-source generation models.

[0013] Secondly, this application provides a general image forgery detection system based on dual-channel difference verification, comprising: The feature encoding module is used to acquire the detection image and the semantic anchor text. A pre-trained visual language model is used to encode the detection image and the semantic anchor text respectively to obtain the global features of the image and the anchor text features. The gradient extraction module is used to determine the similarity score based on the global features of the image and the anchor text features, and to perform backpropagation processing using the similarity score as a virtual loss function to extract the gradient tensors of preset key levels in the model. The signal discrimination module is used to input the gradient tensor into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity discrimination, and output the signal dimension forgery probability. The image generation module is used to generate a textual description of the visual content of the detected image to obtain a target text description, and to perform image generation processing on the target text description to obtain a reference image. The semantic discrimination module is used to extract the visual features of the target image, the text features of the target text description, and the reference visual features of the reference image based on a unified cross-modal shared feature space. Based on the visual features, text features, and reference visual features, the module performs semantic authenticity discrimination to determine the semantic dimension forgery probability. The fusion output module is used to perform weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability based on the dynamic weight coefficients calculated from the global features of the image, so as to obtain the image forgery detection result.

[0014] Thirdly, this application also provides an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the general image forgery detection method based on dual-channel difference verification provided in the first aspect above.

[0015] Fourthly, this application also provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the general image forgery detection method based on dual-channel differential verification provided in the first aspect.

[0016] It is understood that the beneficial effects of the technical solutions provided in the second, third and fourth aspects above can be found in the relevant descriptions in the first aspect above, and will not be repeated here.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a general image forgery detection method based on dual-channel difference verification according to an embodiment of this application; Figure 2 This is a network model framework diagram corresponding to the method shown in the embodiments of this application; Figure 3This is a block diagram of a general image forgery detection system based on dual-channel difference verification, according to an embodiment of this application. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. It should be understood that the specific embodiments described herein are merely for explaining this application and are not intended to limit this application.

[0021] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] Please see Figures 1 to 2 This embodiment provides a general image forgery detection method based on dual-path difference verification, the method specifically includes steps S10-S60.

[0023] Step S10: Obtain the detection image and semantic anchor text, and use a pre-trained visual language model to encode the detection image and the semantic anchor text respectively to obtain the global features of the image and the features of the anchor text. In some embodiments, the input detection image can be preprocessed to normalize its size to a preset standard size and perform pixel value standardization to match the input data distribution of the pre-trained model. Then, the preprocessed image is uniformly divided into several non-overlapping fixed-size image blocks. The multi-channel pixel values ​​of each image block are flattened into a one-dimensional vector, and mapped to a preset-dimensional feature space through a learnable linear projection layer to obtain a block feature vector for each image block. A learnable classification token is concatenated to the beginning of all block feature vectors to form an initial feature sequence containing the classification token and all block features. A learnable positional encoding is added to this sequence to preserve the spatial location information corresponding to each image block. Finally, the feature sequence carrying the positional encoding is input into the image encoder of the pre-trained visual language model, and sequentially passes through a multi-layer transformer encoding layer for self-attention calculation and a feedforward network transformation. The feature vector corresponding to the classification token in the output sequence of the last layer is extracted as the global image feature.

[0024] The global representation vector, obtained by aggregating all image patch information through the self-attention mechanism of a multi-layer transformer, integrates multi-scale information from low-level texture details and statistical properties to high-level semantic content, serving as a unified feature carrier shared by the subsequent dual-path detection process. This processing method, through a hierarchical information aggregation mechanism, enables the obtained global features to possess both low-level signal representation capabilities and high-level semantic expression capabilities. This supports subsequent multi-modal consistency measurement at the semantic level and provides a unified feature benchmark for gradient anomaly analysis in the signal dimension.

[0025] In other embodiments, the encoding of semantic anchor text is performed within the same cross-modal shared feature space as image encoding. The semantic anchor text, as defined in this embodiment, is a pre-selected, fixed textual representation used to characterize the standard semantics of real natural images. It serves as a unified semantic reference for judging image authenticity and remains constant throughout the detection process, requiring no dynamic adjustment based on the input image. When encoding the semantic anchor text, it is first segmented to generate a text token sequence containing a start marker, semantic units, and an end marker, with the sequence length uniformly padded to a preset length. The text token sequence is then converted into word embedding vectors, and text position encoding of the corresponding length is added. This is input into the text encoder of a pre-trained visual language model. After attention operations and feature transformations in the multi-layer transformer encoding layer, the feature vector corresponding to the start marker in the final output sequence is extracted as the anchor text feature.

[0026] It should be noted that during the above encoding process, all parameters of the pre-trained visual language model remain fixed, fully preserving its general representational capabilities learned from massive amounts of natural image-text pairing data. This avoids the risks of knowledge forgetting and overfitting to specific datasets caused by fine-tuning. The anchor text features are vector representations of real semantic standards in a cross-modal shared feature space, residing in the same metric space as the global image features. The semantic matching degree between the two can be directly calculated through vector similarity. This setup provides a numerical metric basis for subsequently constructing the virtual loss function and performing backpropagation gradient extraction, ensuring the stability and reliability of the signal dimension detection results.

[0027] Optionally, the pre-trained visual language model in this implementation can be a Contrastive Language–Image Pre-training (CLIP) model. The CLIP model internally forms a profound representation of the physical laws governing light, shadow, texture, and structure of the natural world. When a forged image generated by an algorithm, even if indistinguishable from reality to the human eye, is input, the image's underlying signal statistics (such as frequency distribution, noise patterns, and pixel correlation) will almost inevitably deviate from the statistical laws of natural images. This deviation may not immediately lead to misclassification during the model's forward propagation, but when the model is forced to determine that the image is real, this underlying conflict will erupt in different levels of the model as abnormal gradients. The goal of the signal difference pathway is to design a dedicated gradient analysis model—the Gradient Meta-Adapter (GMA)—to capture and understand these abnormal gradient patterns. Specific implementations may include the following: 1. Base Model and Anchor Point Settings: Select CLIP's image encoder E img and text encoder E txt As a completely frozen base. Set one or a set of fixed text prompts as absolutely true semantic anchors, such as t real =="areal, high-quality, authentic photograph". This anchor point remains unchanged throughout the process.

[0028] 2. Forward Propagation and Loss Calculation: For any input image x to be tested, firstly through E... img Calculate its image features f img (Including CLS tokens and patch tokens). Meanwhile, through E txt Calculate t real Text features f real Then, calculate the similarity score between them. This score represents how realistic the CLIP model considers the image to be.

[0029] 3. Gradient Extraction (Virtual Backpropagation): Treat the score S as a virtual loss function and backpropagate it to calculate S relative to E. img The gradients of the internal parameters. This backpropagation process does not trigger the optimizer to update any weights of Eimg; its sole purpose is to examine the model's internal state. Instead of capturing gradients from all layers, we selectively detect gradients at the outputs of K key Transformer blocks (e.g., shallow, medium, and deep blocks), capturing the gradient tensors flowing through these locations. These gradient tensors contain a wealth of information: (1) Gradient magnitude: Regions with abnormally large gradient values ​​may correspond to image regions that the model considers to be in the most conflict with the real concept; (2) Gradient direction and distribution: The distribution pattern of the gradient may reveal the specific fingerprint of the forgery algorithm. For example, the upsampling of GAN may lead to a checkerboard gradient pattern, while the denoising process of the diffusion model may lead to a smoother but statistically abnormal gradient field.

[0030] Step S20: Based on the global features of the image and the anchor text features, determine the similarity score, use the similarity score as a virtual loss function to perform backpropagation processing, and extract the gradient tensors of the preset key levels in the model; In some embodiments, the cosine similarity between global image features and anchor text features in a cross-modal shared feature space is calculated as a similarity score representing the true attributes of the image; the similarity score is used as a virtual loss function to perform virtual backpropagation, wherein the backpropagation process does not update any weight parameters of the visual language model, but is only used to trigger the internal gradient response of the visual language model for the true semantic anchor; multi-layer coding blocks of the image encoder in the visual language model are selected as preset key layers, and gradient tensors flowing through each key layer are extracted respectively, wherein the gradient tensor contains gradient magnitude, gradient direction and gradient spatial distribution information.

[0031] Specifically, by calculating the cosine similarity between the global features of the image and the anchor text features in the cross-modal shared feature space, a similarity score is obtained to characterize the degree of matching between the image and the real semantic standard. The virtual loss function is a mechanism design of the signal difference detection path in this scheme. It refers to a virtual objective function that does not undertake the function of optimizing model parameters but is only used to trigger gradient backpropagation. It is different from the real loss in conventional model training. It is not used to update the weight parameters of the pre-trained model, but only exposes the signal response conflicts of the model to the input image through backpropagation.

[0032] In practice, the similarity score is used as the value of the virtual loss function, with the virtual optimization direction being to improve the matching degree between the image and the real semantic anchor point. Backpropagation is performed on the image encoder of the pre-trained visual language model. The entire backpropagation process only completes the calculation and propagation of gradient values, without triggering any parameter update operations or modifying any internal parameters of the image encoder. It only outputs the gradient propagation results between the layers of the model. This processing logic relies on the statistical priors learned by the pre-trained model on massive natural images. Through virtual backpropagation, it transforms the low-level signal deviations that are invisible to the human eye into quantifiable gradient anomalies. It can capture the signal-level flaws of high-fidelity forged images without fine-tuning the base model, while avoiding the knowledge forgetting and overfitting problems of specific generated artifacts caused by model fine-tuning.

[0033] In other embodiments, the extraction of key-level gradient tensors employs a hierarchical, fixed-point acquisition strategy. The preset key levels refer to multiple transformer coding layers pre-selected and distributed at different depths in the image encoder, corresponding to the shallow, middle, and deep stages of feature extraction, respectively. Different depth levels can capture signal anomaly information of different granularities: shallow layers correspond to low-level details such as texture and frequency, middle layers to mid-level features such as structure and edges, and deep layers to high-level features such as global statistical properties. The gradient tensor is the set of gradients of the output parameters of the corresponding layer during backpropagation, carrying information about the model's response pattern to anomalies in the input image signal. During acquisition, gradient capture is performed only at the output of each key coding layer along the backpropagation gradient flow path, extracting the complete gradient tensor corresponding to the output features of the corresponding layer, without needing to collect gradient data from all layers of the entire network.

[0034] The hierarchical selective acquisition method in this embodiment can cover signal anomaly features of different dimensions, comprehensively capture the differentiated artifact fingerprints left by different types of generation models, adapt to various forgery image generation paradigms, and effectively compress the volume of gradient data, reducing the computational cost of subsequent processing. At the same time, the gradient magnitude and distribution pattern of different levels contain the low-level statistical deviation information of the forgery image, which can provide multi-dimensional discrimination basis for subsequent multi-level feature aggregation and authenticity judgment.

[0035] Step S30: Input the gradient tensor into a preset gradient meta-adaptor to perform multi-level feature aggregation and authenticity discrimination, and output the signal dimension forgery probability; In some embodiments, the gradient tensors of each key level are flattened and linearly projected to obtain gradient feature sequences of the corresponding dimensions; learnable positional codes are added to the gradient feature sequences to distinguish the model levels corresponding to the gradients; the gradient feature sequences with added positional codes are input into a Transformer encoder with a preset number of layers for global feature aggregation to obtain aggregated gradient features; the aggregated gradient features are input into a linear classifier to output the signal dimension spoofing probability.

[0036] In one example, the gradient meta-adaptor first performs pre-feature normalization on the input gradient tensors of each key level. The gradient meta-adaptor is used for noise reduction, dimensionality reduction, hierarchical correlation aggregation, and true / false pattern recognition of high-dimensional, multi-source gradient data. It is the core processing unit in the signal difference path that transforms the original gradient information into standardized forgery probabilities. The specific data processing flow is as follows: For the high-dimensional gradient tensor output from each key level, a flattening operation is first performed to convert the multi-dimensional gradient data into a one-dimensional vector form. Then, a parameter-shared linear projection layer uniformly maps all gradient vectors to a preset low-dimensional feature space, obtaining the gradient feature vector corresponding to each key level. Subsequently, according to the encoder depth order of the gradient source, the gradient feature vectors are concatenated into an ordered gradient feature sequence, and a learnable hierarchical position code is added to each position in the sequence to mark the model depth level corresponding to each gradient feature. This processing flow achieves noise reduction and dimensionality reduction of the high-dimensional original gradient through a shared projection layer, filtering out random noise interference in the gradient. At the same time, it retains the deep hierarchical attributes of the gradient by using hierarchical position encoding, avoiding confusion of features at different levels, and providing ordered and low-noise feature input for subsequent cross-level feature aggregation, which can effectively improve the stability and accuracy of the subsequent discrimination process.

[0037] In another example, the gradient meta-adaptor performs multi-level feature aggregation and authenticity discrimination through a multi-level transformer encoding structure. Multi-level feature aggregation refers to the adaptive fusion of gradient information at different depth levels based on a self-attention mechanism. Unlike conventional feature splicing or fixed-weighted fusion methods, this automatically adjusts the attention given to each level of information according to the gradient distribution pattern of the input image. Specifically, the gradient feature sequence carrying the layer position encoding is input into the multi-level transformer encoder. The encoder's internal self-attention mechanism automatically learns the correlation between gradient features at different depth levels, adaptively capturing the linkage patterns of gradient anomalies at each level. It globally integrates information from shallow texture-level gradient deviations and deep statistical-level gradient anomalies, outputting an aggregated feature sequence containing anomaly information from all levels. Subsequently, global pooling is performed on the aggregated feature sequence to obtain a one-dimensional global gradient aggregated feature. This feature is then input into a linear classifier and mapped by an activation function to output the signal dimension as the forgery probability. This processing logic can fully explore the intrinsic correlation between multiple gradients and comprehensively cover the differentiated gradient fingerprints left by different types of forgery generation. It can capture the features of forged images with abnormally significant signals and identify the weak statistical deviations of high-fidelity generated images, thereby improving the generalization and adaptation capability of signal dimension detection to different types of generation models.

[0038] Specifically, since the captured original gradient tensors are high-dimensional and noisy, in order to extract discriminative information from them, the data processing of the designed Gradient Meta-Adapter (GMA) can be as follows: (1) Input: The captured K gradient tensors are first flattened and projected to a lower dimension through a shared linear layer to form a sequence of length K. We add learnable positional encoding to this sequence to distinguish the source level of the gradient. (2) Structure: GMA itself is a small Transformer encoder with 2-4 layers. Its internal self-attention mechanism can learn the complex relationships between gradients. It can integrate gradient information from different depth levels to form a global, comprehensive result. (3) Output: GMA finally outputs a feature vector that aggregates all gradient information. For example, it can be obtained by taking the first position of the output sequence or by average pooling. This feature vector is fed into a simple linear classifier, which finally outputs the spoofing probability Score at the signal level. signal .

[0039] Step S40: Generate a textual description of the visual content of the detected image to obtain a target text description; perform image generation processing on the target text description to obtain a reference image. In some embodiments, the visual content of the detected image is semantically parsed and converted into natural language to generate a target text description covering the subject object, attribute features, spatial relationships and scene information; the target text description is used as a semantic constraint to perform image semantic reconstruction processing to generate a reference image that semantically matches the target text description.

[0040] Specifically, the target text description is a symbolic natural language representation of the visual content of the detected image. It is an intermediate output that converts visual signals into semantic carriers, used to carry core semantic information such as the subject, scene, attributes, and behavior in the image, serving as a semantic benchmark for subsequent logical verification. In the specific processing, the detected image is input into a pre-trained image description generation model with fixed parameters. The model first extracts multi-scale visual features of the image through a built-in visual encoder, and then maps the visual features to the text semantic space through a cross-modal alignment module to obtain a semantic representation aligned with the text representation system. Subsequently, the model generates coherent natural language sentences word by word based on the semantic representation using an autoregressive decoding method, fully covering the core visual elements of the image, and outputs the target text description.

[0041] In this embodiment, the processing procedure converts unstructured image pixel information into structured language symbols, realizing the explicit expression of visual semantics. For real images whose content conforms to objective common sense, the generated text description is logically consistent and accurate in its elements. However, for forged images with semantic errors, the description process may exhibit missing elements, logical contradictions, or vague expressions due to the content violating common cognitive laws. This can initially amplify the forgery flaws at the semantic level, providing a standardized semantic benchmark for comparison in subsequent closed-loop verification.

[0042] In other embodiments, the reference image generation process based on the target text description and the aforementioned textification process together constitute an image-text-image logical verification closed loop. This closed loop is the core mechanism of the semantic difference detection path of this scheme. It refers to constructing a semantic loop verification link through two cross-modal transformations, and using the common sense prior built into the generation model to amplify the semantic deviation of the forged image, which is different from the conventional single-modal semantic analysis method.

[0043] In this embodiment, the generated target text description is input as a generation instruction into a pre-trained text-to-image generation model with fixed parameters. The model first encodes the target text description using a text encoder, extracting semantic elements and constraints. Then, it performs image generation operations based on the semantic features, outputting a reference image corresponding to the text semantics. This reference image is a reference sample reconstructed from the semantic description of the original image. Its generation process relies on the physical common sense and logical cognition learned by the generation model from massive amounts of real data. For compliant real images, the reverse-generated reference image and the original detection image maintain a high degree of consistency in core semantics and main structure. However, for forged images with semantic and logical fallacies, the generation model corrects the abnormal content based on its built-in common sense priors, resulting in a significant semantic deviation between the reference image and the original detection image. This transforms implicit cognitive-level forgery flaws into explicit differences in image features, providing a quantifiable comparison basis for subsequent semantic consistency measurement.

[0044] Step S50: Based on a unified cross-modal shared feature space, extract the visual features of the target image, the text features of the target text description, and the reference visual features of the reference image respectively. Perform semantic authenticity judgment based on the visual features, text features, and reference visual features to determine the semantic dimension forgery probability. In some embodiments, the cross-modal shared feature space is a unified feature metric space learned by a pre-trained visual language model through comparative learning on large-scale image-text pairing data. Feature vectors from different modalities within this space follow the same semantic metric rule. Semantically matching image and text features are close in distance, while semantically contradictory features are far apart, serving as the foundation for cross-modal consistency quantization calculation. During specific data processing, the visual features of the target image directly reuse the global image features generated in the preceding steps, avoiding computational redundancy caused by repeated encoding. The target text description is input into the text encoder of the pre-trained visual language model, and after word segmentation embedding and multi-layer transformer encoding, the global semantic vector of the corresponding text is extracted as text features. A reference image is input into the image encoder of the same pre-trained visual language model, and after a block embedding and multi-layer transformer encoding process identical to that of the target image, reference visual features are extracted. The generation of these three types of features all rely on the same pre-trained model with fixed parameters, ensuring that all features are under a completely consistent metric benchmark. This eliminates modal bias caused by different encoding systems and guarantees the comparability and reliability of subsequent consistency calculation results.

[0045] In other embodiments, the cosine similarity between visual features and text features is calculated as an image-text consistency score; the cosine similarity between visual features and reference visual features is calculated as an image-text consistency score. Figure 1 Consistency score; the consistency score between text and image is compared with the image score. Figure 1 The consistency scores are concatenated into a two-dimensional semantic feature vector, which is input into a multilayer perceptron for authenticity determination and outputs the semantic dimension forgery probability. Specifically, semantic authenticity determination is accomplished by a two-dimensional consistency measurement mechanism. Semantic authenticity determination refers to quantifying the logical self-consistency of image content from two dimensions: cross-modal semantic consistency and cyclic reconstruction consistency. This is used to determine the forgery risk of the image at the semantic level, which is different from the traditional discrimination method that only relies on the underlying pixel features.

[0046] In this embodiment, the cosine similarity between the visual features of the target image and the text features is first calculated to obtain an image-text consistency score, which represents the degree of semantic agreement between the original image and its text description; then, the cosine similarity between the visual features of the target image and the reference visual features is calculated to obtain an image-text consistency score. Figure 1 Image-text consistency score, which characterizes the degree of semantic retention of an image after textual description and reverse reconstruction; the image-text consistency score is compared with the image... Figure 1 The consistency scores are concatenated into a two-dimensional feature vector, which is then input into a multilayer perceptron classifier. After nonlinear transformation and activation function mapping in a fully connected layer, the semantic dimension forgery probability is output. This dual-dimensional measurement mechanism can cover semantic forgery scenarios from two complementary perspectives. Image-text consistency can capture the description mismatch problem caused by chaotic image content. Figure 1Consistency can amplify the semantic deviations in reverse generation of counterfeit content that defies common sense. The combination of the two can comprehensively identify counterfeit images that have no obvious flaws at the pixel level but have semantic logical fallacies, filling the detection gap at the cognitive level in traditional detection methods.

[0047] In one example, a real image exhibits a high degree of bidirectional, logical consistency between its visual content and its accurate linguistic description. AI-generated images, even if they appear good in a one-way (text→image) generation, struggle to maintain logical consistency in this bidirectional, cyclical verification. The goal of the semantic difference pathway is to construct such a logical verification loop of "image→text→image" and quantify the inconsistencies within it using a Multimodal Consistency Module (MCM). The specific implementation is as follows: 1. Image-to-Text Reverse Description: Linguisticization of image content: This involves converting the input visual signal x into a symbolic linguistic description. A powerful, frozen visual-language model (VLM), such as BLIP-2 or LLaVA, can be used as the image descriptor Mcap. 2. Text-to-Image Reverse Generation: Visualization of linguistic description: This involves transforming the text description t generated in the previous step... gen As an instruction, a powerful, frozen text is input into the image generation model M. gen (As in Stable Diffusion), a reference image x is regenerated based on this description. ref If the original image x is real and conforms to common sense, then M gen It can generate an image that is very similar to x in its core semantics; however, if the content of x is incorrect, such as seven fingers, M... gen In trying to generate x ref Sometimes, strange results may occur due to conflicts in its internal prior knowledge, or a more normal image of a five-finger pianist may be generated. This difference itself is an important clue; 3. Multimodal consistency module: This is the core of quantifying logical differences. CLIP's powerful multimodal alignment capabilities are used to calculate two key consistency scores: (l) Image-text consistency (S) I-T ): This measures the semantic similarity between the original image x and its self-generated description tgen. This is achieved by calculating their feature cosine similarity in the CLIP shared space: In the formula, The image-text consistency score, This is the cosine similarity calculation function. For image encoders, XFor the input original image sample, This is a text description generated based on the original image x.

[0048] For a genuine, clear image, this score should be very high. However, for a cluttered or bizarrely forged image, the descriptor may generate inaccurate text, causing the score to decrease.

[0049] (2) Figure Figure 1 S I-I We measure the visual similarity between the original image x and a reference image xref generated based on its description. This is achieved by calculating their cosine similarity in the CLIP visual feature space: In the formula, For the image Figure 1 Consistency score, A reference image generated using the original image as the prompt text.

[0050] 4. Final judgment: The two consistency scores S are... I-T and S I-I Concatenate them into a two-dimensional feature vector [S] I-T ,S I-I This vector is fed into a very small, learnable multi-layer perceptron (MLP), which is trained to learn how to determine whether an image has a semantic problem based on the combination of these two scores, and finally outputs the semantic-level forgery probability.

[0051] Step S60: Based on the dynamic weight coefficients calculated from the global features of the image, perform a weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability to obtain the image forgery detection result.

[0052] In some embodiments, global image features are input into a pre-gated network. After feature mapping is performed through a linear transformation layer, the output value is constrained to a normalized numerical range by a non-linear activation function to obtain dynamic weight coefficients in single-scalar form. The dynamic weight coefficients are used as weighting coefficients for the signal dimension forgery probability, and the complementary values ​​corresponding to the dynamic weight coefficients are used as weighting coefficients for the semantic dimension forgery probability. A weighted summation operation is performed on the signal dimension forgery probability and the semantic dimension forgery probability to output the image forgery detection result. The trainable parameters of the pre-gated network, the gradient aggregation fitting parameters corresponding to the signal dimension detection, and the multilayer perceptron parameters corresponding to the semantic dimension detection are updated end-to-end through backpropagation using a unified classification loss function during the model training phase.

[0053] It should be noted that the dynamic weight coefficients are dual-path credibility weights dynamically calculated based on the input image's own features. This differs from the fixed weights used in traditional static fusion. The dynamic weight coefficients can automatically adjust the contribution ratio of signal dimension detection and semantic dimension detection results according to the type and severity of image forgery flaws. Specifically, the data processing flow is as follows: the global image features obtained in the preceding steps are input into a lightweight adaptive gating network, which consists of a linear transformation layer and a nonlinear activation function connected in series. First, the high-dimensional global image features are compressed and linearly mapped using the linear transformation layer, outputting a one-dimensional intermediate calculated value. Then, the intermediate value is mapped to a normalized numerical range using the nonlinear activation function to obtain the weight coefficients corresponding to the signal dimension. Finally, the weight coefficients corresponding to the semantic dimension are calculated through numerical complementation. These two sets of weight coefficients together constitute the dynamic weight coefficients. This processing logic uses global image features that condense the global texture, style, and semantic attributes of an image as the basis for weight determination. It can quickly perceive the distribution characteristics of forgery clues in the input image, automatically increase the weight of the signal path for images with significant signal-level artifacts, and automatically increase the weight of the semantic path for images with obvious semantic logic deviations. This avoids the performance shortcomings of static weighting strategies in single forgery type scenarios and ensures that reasonable detection and discrimination criteria can be obtained in different scenarios.

[0054] In some embodiments, weighted fusion calculation sums the dual-path discrimination results according to dynamic weight coefficients, outputting the final image forgery detection result. Weighted fusion calculation refers to the process of weighted integration of the signal dimension forgery probability and the semantic dimension forgery probability based on dynamically allocated path weights. Specifically, the signal dimension forgery probability is multiplied by its corresponding weight coefficient to obtain the weighted contribution value of the signal path; the semantic dimension forgery probability is multiplied by its corresponding weight coefficient to obtain the weighted contribution value of the semantic path; the two sets of weighted contribution values ​​are summed to obtain the final forgery probability characterizing the overall forgery risk of the image, which serves as the core quantitative basis for the image forgery detection result. This fusion mechanism can organically integrate two different dimensions of forgery clues: microscopic signal anomalies and macroscopic semantic deviations. It retains the detection advantages of each path while achieving the optimal ratio of the two types of clues through dynamic weights. Simultaneously, this fusion process can be jointly trained end-to-end with the front-end dual-path discrimination module, enabling synergistic optimization between the weight allocation logic and the discrimination characteristics of the dual paths, thereby improving the overall detection scheme's discrimination accuracy and generalization ability for multiple types of generated forged images.

[0055] In the method steps provided in the above embodiments, a two-dimensional parallel detection system of a pre-trained visual language model is used. In the signal detection dimension, backpropagation is performed by using the encoding similarity between the detected image and the semantic anchor text as a virtual loss to extract the gradient tensor of key levels inside the model. With the help of the physical statistical priors learned by the pre-trained model in massive natural images, the deviation between the image's underlying signal and the natural image's regularity is transformed into a quantifiable gradient anomaly pattern. Without relying on explicit pixel-level artifact features, it can capture statistical regularity shifts in high-fidelity generated images that are difficult to observe directly. Combined with the gradient meta-adaptor to perform feature aggregation and discrimination on multi-level gradient tensors, it can integrate the anomaly information captured by different depth levels of the model, taking into account both the subtle deviations at the shallow texture frequency level and the structural signal anomalies at the deep feature level, further enhancing the comprehensiveness and accuracy of signal dimension detection. At the same time, since the parameters of the pre-trained model remain fixed, the detection logic will not overfit to the features of a specific generation model, and can adapt to more unknown types of generation forgery scenarios.

[0056] Furthermore, in the semantic detection dimension, a complete logical verification loop is constructed through image content textual description and reverse image generation. This leverages the common sense priors built into the generation model to amplify semantic logical deviations in forged images. After description generation and reverse reconstruction, high semantic consistency is maintained. Forged images with semantic logical deviations will exhibit significant semantic shifts during reverse generation due to the common sense corrections from the generation model. Based on a unified cross-modal shared feature space, the consistency measurement of multi-source features is completed, eliminating measurement biases caused by modal differences. This accurately quantifies the semantic fit between the original image, description text, and reference image, achieving object... It effectively identifies forged images that have no obvious flaws at the pixel level but whose content does not conform to physical common sense and event logic. On this basis, it generates dynamic weight coefficients based on global image features and performs adaptive weighted fusion of the two detection results. It can automatically match the weight ratio of signal dimension and semantic dimension according to the inherent characteristics of the input image such as texture complexity and semantic clarity, so that the two types of detection clues form a complementary effect, avoiding the performance shortcomings of static weighting strategy in specific forgery scenarios. Finally, it achieves comprehensive detection of synthetic forged images of different types and different flaw types, and maintains excellent detection accuracy and generalization performance in test scenarios composed of multiple generative models.

[0057] Please see Figure 3 , Figure 3 A block diagram of the general image forgery detection system based on dual-channel difference verification provided in this embodiment is shown. The general image forgery detection system 200 based on dual-channel difference verification includes: The feature encoding module 210 is used to acquire the detection image and the semantic anchor text, and to encode the detection image and the semantic anchor text respectively using a pre-trained visual language model to obtain the global features of the image and the anchor text features. The gradient extraction module 220 is used to determine the similarity score based on the global features of the image and the anchor text features, and to perform backpropagation processing using the similarity score as a virtual loss function to extract the gradient tensor of the preset key level in the model. The signal discrimination module 230 is used to input the gradient tensor into a preset gradient element adapter to perform multi-level feature aggregation and authenticity discrimination, and output the signal dimension forgery probability. The image generation module 240 is used to generate a textual description of the visual content of the detected image to obtain a target text description, and to perform image generation processing on the target text description to obtain a reference image; The semantic discrimination module 250 is used to extract the visual features of the target image, the text features of the target text description, and the reference visual features of the reference image based on a unified cross-modal shared feature space, and to perform semantic authenticity discrimination based on the visual features, text features, and reference visual features to determine the semantic dimension forgery probability. The fusion output module 260 is used to perform weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability based on the dynamic weight coefficients calculated from the global features of the image, so as to obtain the image forgery detection result.

[0058] The general image forgery detection system provided in this embodiment constructs a detection architecture that operates in parallel across signal and semantic dimensions. On one hand, it uses the similarity score between global image features and anchor text features as a virtual loss to perform backpropagation, extracting gradient tensors at key model levels and a gradient meta-adaptor to complete multi-level feature aggregation and discrimination. This allows it to capture abnormal patterns at the low-level signal level of images by relying on the inherent representation of statistical regularities in natural images by a pre-trained visual language model, without relying on explicit pixel artifact features. It maintains stable detection and discrimination capabilities even when facing high-fidelity forged images. On the other hand, it constructs a detection architecture through textual description of image content and reverse image generation. The verification loop completes the consistency measurement of multi-source features within a unified cross-modal shared feature space. It can judge the logical rationality of image content from a macro-semantic level, covering forgery scenarios where there are no obvious flaws at the pixel level but the content does not conform to physical common sense and event logic. At the same time, it generates dynamic weight coefficients based on global image features and performs adaptive weighted fusion of the two discrimination results. It can match the weight ratio of different detection dimensions according to the characteristics of the input image, integrate two types of discrimination information, namely micro signal anomalies and macro semantic deviations, and achieve comprehensive detection of different types of synthetic forgery images. It can maintain excellent detection accuracy and generalization performance in the test scenario of multi-source generation model.

[0059] It is understood that, in the implementation of the general image forgery detection system 200 based on dual-channel difference verification in this embodiment, each module operates as described above. Figure 1 The steps of a general image forgery detection method based on dual-channel difference verification in the corresponding embodiment, and the technical effects it can achieve, can be found in the above description. Figure 1 The technical effects achieved in the corresponding embodiments will not be elaborated upon here.

[0060] Please see Figure 4 , Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device includes at least a server 500, wherein the server 500 includes a processor 501, a memory 502, and a computer program 503 stored in the memory 502 and executable on the processor 501, such as a program for a general image forgery detection method based on dual-channel difference verification. When the processor 501 executes the computer program 503, it implements the steps of the general image forgery detection method based on dual-channel difference verification in the above embodiments, for example... Figure 1 Steps S10 to S50 correspond to the embodiments. Alternatively, the processor 501 executes the computer program 503 to implement the above. Figure 3 For details on the functions of each module in the corresponding embodiments, please refer to [link / reference]. Figure 3 The relevant descriptions in the corresponding embodiments are not repeated here.

[0061] For example, computer program 503 can be divided into one or more units, one or more units are stored in memory 502 and executed by processor 501 to complete the technical solution provided in the above embodiments. One or more units can be a series of computer program instruction segments capable of performing a specific function, which are used to describe the execution process of computer program 503 in server 500.

[0062] The electronic device may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will understand that... Figure 4 This is merely an example of server 500 in an electronic device and does not constitute a limitation on server 500. It may include more or fewer components than shown, or combine certain components, or different components. For example, a turntable terminal device may also include input / output terminal devices, network access terminal devices, buses, etc.

[0063] The processor 501 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0064] The memory 502 can be an internal storage unit of the server 500, such as the server 500's hard drive or memory. The memory 502 can also be an external storage terminal device of the server 500, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the server 500. Furthermore, the memory 502 can include both internal storage units and external storage terminal devices of the server 500. The memory 502 is used to store computer programs and other programs and data required by the turntable terminal device. The memory 502 can also be used to temporarily store data that has been output or will be output.

[0065] In some embodiments, a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the general image forgery detection method based on dual-channel difference verification as described in the above embodiments.

[0066] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0067] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0068] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects and not to describe a particular order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, it may include a series of steps or units, or optionally, steps or units not listed, or other steps or units inherent to these processes, methods, products, or devices.

[0069] The accompanying drawings show only the portions relevant to this application, not all of them. Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations may be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subprogram, etc.

[0070] The terms “component,” “module,” “system,” “unit,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a unit can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, and / or distributed between two or more computers. Furthermore, these units can be executed from various computer-readable media on which various data structures are stored. Units can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from a second unit interacting with another unit between a local system, a distributed system, and / or a network; for example, the Internet interacting with other systems via signals).

[0071] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0072] Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The reference to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily indicate the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0073] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0074] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

Claims

1. A general image forgery detection method based on dual-channel difference verification, characterized in that, include: The detection image and semantic anchor text are acquired, and a pre-trained visual language model is used to encode the detection image and the semantic anchor text respectively to obtain the global features of the image and the features of the anchor text. Based on the global features of the image and the anchor text features, a similarity score is determined, and the similarity score is used as a virtual loss function to perform backpropagation processing to extract the gradient tensors of preset key levels within the model. The gradient tensor is input into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity discrimination, and the output signal dimension forgery probability is obtained. The detected image is subjected to textual description of visual content to obtain a target text description, and the target text description is subjected to image generation processing to obtain a reference image; Based on a unified cross-modal shared feature space, visual features of the target image, text features of the target text description, and reference visual features of the reference image are extracted respectively. Semantic authenticity is determined based on visual features, text features, and reference visual features to determine the semantic dimension forgery probability. Based on the dynamic weight coefficients calculated from the global features of the image, a weighted fusion calculation is performed on the forgery probability of the signal dimension and the forgery probability of the semantic dimension to obtain the image forgery detection result.

2. The general image forgery detection method based on dual-channel difference verification according to claim 1, characterized in that, The step of determining a similarity score based on the global features of the image and the anchor text features, and using the similarity score as a virtual loss function to perform backpropagation processing to extract the gradient tensors of preset key levels within the model includes: Calculate the cosine similarity between the global features of the image and the anchor text features in the cross-modal shared feature space, and use it as a similarity score representing the true attributes of the image; The similarity score is used as a virtual loss function to perform virtual backpropagation, wherein the backpropagation process does not update any weight parameters of the visual language model, but is only used to trigger the internal gradient response of the visual language model for the real semantic anchor point. The multi-layer coding blocks of the image encoder in the visual language model are selected as preset key layers, and the gradient tensors flowing through each key layer are extracted respectively. The gradient tensors contain gradient magnitude, gradient direction and gradient spatial distribution information.

3. The general image forgery detection method based on dual-channel difference verification according to claim 2, characterized in that, The step of inputting the gradient tensor into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity determination, and outputting the signal dimension forgery probability, includes: The gradient tensors of each key level are flattened and linearly projected to obtain the gradient feature sequences of the corresponding dimensions. Learnable positional codes are added to the gradient feature sequences to distinguish the model layers corresponding to the gradients; The gradient feature sequence with added position encoding is input into a Transformer encoder with a preset number of layers for global feature aggregation to obtain aggregated gradient features; The aggregated gradient features are input into a linear classifier, which outputs the signal dimension spoofing probability.

4. The general image forgery detection method based on dual-channel difference verification according to claim 1, characterized in that, The steps of generating a textual description of the visual content of the detected image to obtain a target text description, and performing image generation processing on the target text description to obtain a reference image, include: Semantic parsing and natural language conversion are performed on the visual content of the detected images to generate target text descriptions covering the main object, attribute features, spatial relationships and scene information; The target text description is used as a semantic constraint to perform image semantic reconstruction processing, generating a reference image that semantically matches the target text description.

5. The general image forgery detection method based on dual-channel difference verification according to claim 4, characterized in that, The step of determining the semantic dimension forgery probability by performing semantic authenticity judgment based on visual features, text features, and reference visual features includes: Calculate the cosine similarity between the visual features and the text features, and use it as the image-text consistency score; Calculate the cosine similarity between the visual feature and the reference visual feature, and use it as the graph-to-graph consistency score; The image-text consistency score and the image-image consistency score are concatenated into a two-dimensional semantic feature vector, which is then input into a multilayer perceptron for authenticity determination, and the semantic dimension forgery probability is output.

6. The general image forgery detection method based on dual-channel difference verification according to claim 1, characterized in that, The step of performing a weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability based on the dynamic weight coefficients calculated from the global features of the image to obtain the image forgery detection result includes: The global features of the image are input into a preset gated network. After feature mapping is performed through a linear transformation layer, the output value is constrained to a normalized numerical range through a nonlinear activation function to obtain dynamic weight coefficients in single scalar form. The dynamic weight coefficient is used as the weighting coefficient for the signal dimension forgery probability, and the complementary value corresponding to the dynamic weight coefficient is used as the weighting coefficient for the semantic dimension forgery probability. A weighted summation operation is performed on the signal dimension forgery probability and the semantic dimension forgery probability to output the image forgery detection result.

7. The general image forgery detection method based on dual-channel difference verification according to claim 6, characterized in that, The trainable parameters of the preset gating network, the gradient aggregation fitting parameters corresponding to signal dimension detection, and the multilayer perceptron parameters corresponding to semantic dimension detection are updated through end-to-end backpropagation using a unified classification loss function during the model training phase.

8. A general image forgery detection system based on dual-channel difference verification, characterized in that, include: The feature encoding module is used to acquire the detection image and the semantic anchor text. A pre-trained visual language model is used to encode the detection image and the semantic anchor text respectively to obtain the global features of the image and the anchor text features. The gradient extraction module is used to determine the similarity score based on the global features of the image and the anchor text features, and to perform backpropagation processing using the similarity score as a virtual loss function to extract the gradient tensors of preset key levels in the model. The signal discrimination module is used to input the gradient tensor into a preset gradient meta-adaptor for multi-level feature aggregation and authenticity discrimination, and output the signal dimension forgery probability. The image generation module is used to generate a textual description of the visual content of the detected image to obtain a target text description, and to perform image generation processing on the target text description to obtain a reference image. The semantic discrimination module is used to extract the visual features of the target image, the text features of the target text description, and the reference visual features of the reference image based on a unified cross-modal shared feature space. Based on the visual features, text features, and reference visual features, the module performs semantic authenticity discrimination to determine the semantic dimension forgery probability. The fusion output module is used to perform weighted fusion calculation on the signal dimension forgery probability and the semantic dimension forgery probability based on the dynamic weight coefficients calculated from the global features of the image, so as to obtain the image forgery detection result.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of a general image forgery detection method based on dual-channel differential verification as described in any one of claims 1-7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the steps of any one of the general image forgery detection methods based on dual-channel difference verification as described in claims 1-7.

Images