Multi-modal false news detection method based on decoupling and cross-modal clue mining

Abstract

The invention belongs to the technical field of artificial intelligence and information security crossing, and discloses a multi-modal false news detection method based on decoupling and cross-modal clue mining, and the method comprises the steps: employing a pre-training comparison model to extract text and image features in news, and mapping the text and image features to a unified embedding space; designing a multi-modal decoupling automatic encoder, decomposing each modal feature into modal common and modal specific representation, and obtaining a pure decoupling feature through confrontation constraint and reconstruction loss; a cross-modal clue mining module is constructed, modeling complementation is carried out from non-unhooked features, and conflicting semantic clues are enhanced; a self-adaptive attention aggregation module is introduced, and the modal features and the semantic clues are dynamically weighted and fused; and inputting the aggregated representations into a classifier for false news identification. According to the method, through modal decoupling and multi-source clue modeling, modal redundancy and specific information are effectively separated, association and conflicts between images and texts are deeply mined, and the detection accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of the intersection of artificial intelligence and information security, and in particular to a multimodal fake news detection method based on decoupling and cross-modal clue mining, which can be applied to scenarios such as social network content review, online rumor identification, public opinion analysis, and intelligent security. Background Art

[0002] With the rapid development of social media, multimodal content such as images, text, and video has become the mainstream form of information dissemination. However, false news and rumors are also rapidly spreading through multimodal means, posing a serious threat to social stability, public safety, and the public opinion environment. Traditional false information detection methods, which are mostly text-based, have difficulty effectively capturing the semantic deviations and complementary relationships between images and text, and are less effective in identifying highly fabricated content and cross-modal inconsistencies.

[0003] In recent years, multimodal fake news detection methods have become a research hotspot, with attempts to improve detection performance by fusing image and text information. However, existing methods generally face the following challenges: (1) Modal heterogeneity and redundant interference problems: Different modalities have natural differences in semantic structure and representation, and simple fusion is prone to introduce noise and redundant information; (2) Representation confusion problem: Image and text features contain both shared semantics that describe the same thing and their own unique semantics that complement each other. If they are mixed and encoded indiscriminately, it will lead to feature representation confusion and reduce discriminative power; (3) Insufficient key clue extraction ability: Fake news often uses inconsistencies between images and text or subtle semantic conflicts to disguise itself. Existing methods find it difficult to deeply mine such highly discriminative cross-modal clues; (4) Lack of interpretability mechanism: Most deep models are "black box" structures and lack visual explanations of prediction results, which affects the credibility and controllability of the models in practical applications.

[0004] Therefore, there is an urgent need for a new multimodal fake news detection method that can effectively address the above challenges and has the capabilities of modal decoupling, deep cross-modal clue mining and interpretability to improve the accuracy, robustness and practical application value of detection. Summary of the Invention

[0005] Based on this, the present invention proposes a multimodal fake news detection method based on decoupling and cross-modal clue mining, aiming to solve the problems of existing multimodal fake news detection methods in modal heterogeneity, representation confusion, insufficient key clue mining and lack of interpretability, which make fake news difficult to detect.

[0006] In a first aspect, the present invention provides a multimodal fake news detection method based on decoupling and cross-modal clue mining, comprising the following steps: Step 1: Modal feature extraction: For multimodal news data containing images and text, a pre-trained contrastive learning model is used to synchronously map the multimodal news data of images and text into a unified embedding space to generate aligned modal feature representations; Step 2: Multimodal feature decoupling: The aligned modal feature representations are input into a multimodal decoupling autoencoder. By adversarial similarity constraints and difference constraints, the feature representation of each modality is decomposed into shared features common to the modalities and specific features unique to the modality. Step 3: Cross-modal clue mining: Based on the decoupled shared features and specific features, the cross-modal interaction module is used to model the semantic association between images and texts to mine complementary, reinforcing, and conflicting semantic clues and generate cross-modal interaction features. Step 4: Adaptive attention aggregation: Introduce an adaptive attention aggregation module to dynamically weight the modal feature representation, shared features, specific features, and cross-modal interaction features to generate a fused representation that can highlight key discriminant information; Step 5, false news classification and identification: The fusion representation is input into the classifier, and the model is trained by jointly optimizing the classification loss and the decoupling loss to achieve the authenticity classification of the multimodal news data.

[0007] As an optional implementation of the first aspect of the present application, in step 1, it specifically includes: using a pre-trained contrastive language-image model as the contrastive learning model, and using its text encoder and image encoder to extract text feature vectors and image feature vectors respectively; inputting the text feature vectors and image feature vectors into a linear mapping layer, transforming them into a unified low-dimensional embedding space to further eliminate the distribution differences between modalities, and normalizing the mapped feature representations.

[0008] As an optional implementation of the first aspect of the present application, in step 2, the multimodal decoupling autoencoder is composed of a shared encoder and a modality-specific encoder; the shared encoder is used to extract common shared features from the feature representations of different modalities; the modality-specific encoder is used to learn unique specific features from the feature representations of each modality; and by introducing a reconstruction decoder, the original input features are reconstructed within the modality and across the modalities to ensure the information integrity of the decoupled features.

[0009] As an optional implementation of the first aspect of the present application, in step 2, the adversarial similarity constraint is implemented by introducing a discriminator, which is used to distinguish the modal sources of shared features and drive the shared encoder to generate a common representation that is consistent across modalities and cannot distinguish the source through adversarial training; the difference constraint is implemented by calculating the cosine similarity between shared features and specific features to ensure the semantic difference between shared features and specific features.

[0010] As an optional implementation of the first aspect of the present application, in step 3, the cross-modal clue mining specifically includes: for shared features, using a dual-stream gated attention mechanism to achieve text-guided image shared feature enhancement and image-guided text shared feature enhancement; for specific features, using a bidirectional interactive network to achieve text-guided image-specific semantic completion and image-guided text-specific semantic completion; the enhanced shared features and the completed specific features are spliced ​​and fused to form a cross-modal interactive feature that comprehensively reflects the semantic consistency and conflict of the image and text.

[0011] As an optional implementation of the first aspect of the present application, in step 4, the adaptive attention aggregation module specifically includes: a multi-head attention mechanism, which is used to perform preliminary aggregation of the spliced ​​modal features and interaction features; a modal confidence assessment network, which is used to dynamically evaluate the contribution of each modality and interaction feature in the current discrimination task and generate corresponding confidence weights; and weighted summation of each feature is performed in combination with the confidence weight to generate the final fusion representation.

[0012] As an optional implementation of the first aspect of the present application, in step 5, the classifier is a multi-layer perceptron, and the cross-entropy loss function is used as the classification loss; the goal of the model training is the weighted sum of the cross-entropy loss and the decoupling loss of the multimodal decoupling autoencoder, and the model is end-to-end jointly optimized through the back-propagation algorithm.

[0013] As an optional implementation of the first aspect of the present application, the method also includes step 6, explainability analysis: in order to provide a visual explanation of the decision-making process of the model, an explainability module based on the graph attention mechanism is introduced; by analyzing the attention weight distribution in the adaptive attention aggregation module, the key image areas, key text fragments and interaction paths between modalities that contribute most to the classification results are identified and output, providing a transparent decision-making basis for content review and public opinion analysis.

[0014] In a second aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0015] In a third aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. Pure, high-quality feature representation: Through a multimodal decoupling autoencoder, the shared semantics and modality-specific semantics in image and text information are effectively separated, significantly reducing redundant interference and representation confusion between modalities, providing a pure and highly expressive feature foundation for subsequent clue mining.

[0017] 2. In-depth cross-modal clue mining: This invention not only focuses on the alignment between modalities, but also deeply explores the complementary, reinforcing, and conflicting relationships between images and text through the interaction of decoupled features. It can effectively identify advanced camouflage methods such as inconsistencies between images and text that are common in fake news, significantly improving detection accuracy.

[0018] 3. Robust dynamic feature fusion: The adaptive attention aggregation mechanism can dynamically adjust the contribution weights of various modal features and interaction cues based on the characteristics of different news samples, enabling the model to focus on the most discriminative information, thereby improving the model's robustness in diverse and complex scenarios.

[0019] 4. Reliable Explainability: This paper designs an explainability module that can transform the model's "black box" decision-making process into user-friendly visualization results, clearly displaying the key graphical evidence that the model relies on to make judgments, greatly enhancing the model's transparency, credibility, and usability in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of a multimodal fake news detection method based on decoupling and cross-modal clue mining according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the model architecture of a multimodal fake news detection method based on decoupling and cross-modal clue mining in an embodiment of the present invention.

[0021] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0023] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the application can be implemented in a sequence other than those illustrated or described here. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated before and after are in a kind of "or" relationship. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically limited.

[0024] Example 1 See also Figure 1 and Figure 2 , this paper proposes a multimodal fake news detection method based on decoupling and cross-modal clue mining. The overall process of this method is as follows Figure 1 As shown, the model architecture is as Figure 2 shown.

[0025] Step 1: Modal feature extraction: For multimodal news data containing images and text, a pre-trained contrastive learning model is used to synchronously map the multimodal news data of images and text into a unified embedding space to generate aligned modal feature representations; This step focuses on two modal information types: news text and images. It uses the pre-trained contrastive learning model CLIP (Contrastive Language-Image Pre-training) to simultaneously extract multimodal features and map them to a unified semantic embedding space to alleviate the heterogeneity problem between different modal features and provide high-quality input representation for subsequent multimodal fusion and decoupling.

[0026] Step 1.1: Image-text feature extraction. Assume that the input news sample is a pair of text-image modal data: (T, V), where T represents the text content and V represents the corresponding news image. Input the text T and image V into the text encoder (CLIP-T) and image encoder (CLIP-V) of the pre-trained CLIP model respectively to obtain the corresponding text feature vector and image feature vector , the specific calculation is as follows: (1) in, , , The vector dimension of the CLIP encoder output.

[0027] Step 1.2: Unify the embedding space mapping. After obtaining the text feature vector output by the CLIP encoder and image feature vector Finally, in order to better adapt to the subsequent multimodal fusion module and eliminate the residual distribution differences between the modalities, a linear mapping layer (fully connected layer) is designed to transform the two modal features and map them to a unified low-dimensional embedding space. ,in The specific mapping process is as follows: (2) in, , are the weight matrices for text and image respectively; , are the corresponding bias vectors respectively; Represents a nonlinear activation function, such as ReLU or GeLU; , is the feature representation of text and image after mapping. and Perform normalization: (3) in and Represents the normalized text features and image features.

[0028] This linear mapping layer can adapt to different data sets and task requirements, further narrow the distribution differences between the two modalities through parameter learning, and improve the subsequent cross-modal fusion and decoupling effects.

[0029] Step 2: Multimodal feature decoupling: The aligned modal feature representations are input into a multimodal decoupling autoencoder. By adversarial similarity constraints and difference constraints, the feature representation of each modality is decomposed into shared features common to the modalities and specific features unique to the modality. This step aims to extract and map the text features in step 1 to a unified embedding space. and image features Perform deep decoupling to extract shared (common) features and modality-specific (unique) features between modalities, thereby reducing redundant information and improving the accuracy and interpretability of cross-modal fusion.

[0030] The multimodal decoupled autoencoder consists of two sub-encoders: (1) shared encoder Responsible for extracting common feature representations between different modalities; (2) Modality-specific encoder and They are responsible for extracting unique feature representations for text and image modalities, respectively. All encoders use a two-layer multi-layer perceptron (MLP) with a GeLU activation function.

[0031] Step 2.1: Feature Mapping. For text features and image features , mapped into shared representation and specific representation respectively: (4) in, , Represent the shared feature representations of text and image respectively; , Representing the specific features of text and image respectively; 、 and represents the learnable parameters of the corresponding encoder.

[0032] Step 2.2: Decoupling constraints. To ensure that the shared representation and the specific representation are not redundant and the representation is effective, the following constraints are adopted: Difference Loss ensures that shared representations and specific representations are semantically distinct and prevents information overlap: (5) in It represents the cosine similarity between shared features and unique features, i.e., the difference constraint.

[0033] Adversarial Similarity Loss, by introducing the discriminator Distinguishing the source modality of shared representations drives the shared encoder to generate common representations that are consistent across modalities.

[0034] (6) in represents the adversarial similarity constraint, is the discriminator parameter; is the true value of the modality label; the discriminator attempts to accurately determine the modality source of the shared representation, and the encoder performs reverse optimization to make the shared representation indistinguishable from the modality source.

[0035] Step 2.3: Reconstruction loss. Design the reconstruction decoder and , using shared representation and specific representation to reconstruct the original input features and achieve information integrity assurance: (1) Intramodal reconstruction: (7) in and Represent the reconstructed semantic representations within text and image modalities respectively; (2) Cross-modal reconstruction: (8) in and Represents the reconstructed semantic representation of text and image across modalities respectively; The reconstruction loss is defined as the sum of the mean squared errors between the original input features and the reconstructed features: (9) in is the reconstruction loss; The combined difference constraint, adversarial similarity constraint, and reconstruction loss constitute the training objective of the multimodal decoupled autoencoder: (10) Through the joint optimization of this loss, the model can explicitly distinguish between shared and specific features, effectively alleviate modal redundancy and representation confusion, and enhance the clarity and expressiveness of cross-modal representations.

[0036] Step 3: Cross-modal clue mining: Based on the decoupled shared features and specific features, the cross-modal interaction module is used to model the semantic association between images and texts to mine complementary, reinforcing, and conflicting semantic clues and generate cross-modal interaction features. This step aims to design a cross-modal Clues Mining (CCM) module based on the shared features and modality-specific features decoupled in step 2, to achieve deep fusion and mutual complementation between text and image modalities, and to mine potential semantic clues and inconsistent information in multimodal fake news. Suppose the shared features and specific features of the text modality are and ; The shared features and specific features of image modalities are and .

[0037] Step 3.1: Modality-Shared Semantic Enhancement: Utilize shared features for cross-modal complementary semantic enhancement, using a dual-stream gated attention mechanism to guide the enhancement of image-shared features using text-shared features, and vice versa.

[0038] Text-guided image-shared feature enhancement: (11) Image-guided text-shared feature enhancement: (12) in, is the sigmoid activation function, which serves as the gate controller; 、 is a learnable parameter; represents vector concatenation; ⊙ represents element-by-element product. This mechanism can filter out irrelevant information and improve the expression of shared semantics.

[0039] Step 3.2: Modality-specific semantic completion: Based on modality-specific features, a bidirectional interaction network is used to perform text-guided image-specific completion and image-guided text-specific completion, capturing the complementary relationship between multimodal unique semantics.

[0040] Text-guided image-specific completion: (13) Image-guided text-specific completion: (14) in, , , , is a learnable parameter; is the hyperbolic tangent activation function.

[0041] Step 3.2: Cross-modal cue fusion. Concatenate the enhanced shared semantics and the completed specific semantics to form a comprehensive cross-modal cue representation: (15) in, , is a learnable parameter. This fusion representation comprehensively reflects the semantic consistency and potential conflicts between the two modalities, which helps to detect the typical inconsistency between images and text in fake news.

[0042] Step 4: Adaptive attention aggregation: Introduce an adaptive attention aggregation module to dynamically weight the modal feature representation, shared features, specific features, and cross-modal interaction features to generate a fused representation that can highlight key discriminant information; First, map each modal feature to a unified dimension , recorded as: text modality feature ; Image modality features ; Graphics and text interaction modal features (Generated by the CCM module). Concatenate them into a joint representation matrix: (16) Introduce a multi-head attention mechanism for preliminary feature aggregation, each attention head Using independent parameters , the calculation of attention is expressed as: (17) in represents the hth attention head, 、 、 . Will eventually After the attention heads are spliced, the fusion features are obtained through linear mapping : (18) in () indicates the splicing operation, Represents the parameter matrix of the linear layer.

[0043] Step 4.2: Modal Confidence Evaluation Network (MCEN). In order to adaptively measure the discriminative contribution of different modalities in the current context, the modal confidence network is introduced to Predict confidence weights separately ,satisfy: (19) The implementation is a two-layer perceptron with shared weights: (20) in Indicates modality The confidence weight of Indicates modality The scoring function is: (twenty one) Step 4.3: Weighted fusion representation. Combined with modal confidence and multi-head attention output , the final fusion representation Defined as: (twenty two) in, is a hyperparameter used to balance the proportion between attention aggregation results and explicit weighted fusion, usually The final fusion feature It is input into the classification module to determine the authenticity label of the multimodal input content.

[0044] Step 5, false news classification and identification: The fusion representation is input into the classifier, and the model is trained by jointly optimizing the classification loss and the decoupling loss to achieve the authenticity classification of the multimodal news data.

[0045] After completing the cross-modal feature fusion, a unified multimodal embedding representation is obtained, which is recorded as ,in Represents the dimension of the fused embedding vector. To distinguish true from false for the input news sample, the embedding vector is input into the Multi-Layer Perceptron (MLP) classifier for prediction.

[0046] Step 5.1: Multilayer Perceptron structure design. Assume that the MLP classifier consists of Layers of fully connected layers, each of which consists of a weight matrix and the bias vector parameterization, . No. The output of the layer is represented as: (twenty three) in, , that is, the fusion features of the input; Represents a nonlinear activation function (such as ReLU, GELU); is the final output representation. The final output is Layer, get the predicted probability of each category: (twenty four) in, represents the predicted probability distribution of the two categories "false" and "true"; and for Classification layer parameters.

[0047] Step 5.2: Loss function design. In order to train the classification model, the cross-entropy loss function is used to measure the difference between the predicted probability and the true label. Given a sample with a true label of (using one-hot encoding), the loss function is defined as follows: (25) in, represents the true label of the i-th sample; Represents the prediction result of the i-th sample.

[0048] Step 5.3: Model optimization and training. Use the backpropagation algorithm and an optimizer (such as Adam) to iteratively update all trainable parameters, minimizing cross-entropy loss and improving classification performance. Early stopping is performed on the validation set throughout the training process to prevent overfitting.

[0049] Optionally, step 6, explainability analysis, is also included: in order to provide a visual explanation of the decision-making process of the model, an explainability module based on the graph attention mechanism is introduced; by analyzing the attention weight distribution in the adaptive attention aggregation module, the key image areas, key text fragments and interaction paths between modalities that contribute most to the classification results are identified and output, providing a transparent decision-making basis for content review and public opinion analysis.

[0050] To enhance the model's interpretability and credibility, we designed an explanation method based on the Graph Attention Mechanism to mine the node features, modal clues, and propagation paths that play a key role in the model's discriminative process. This method specifically includes three submodules: attention weight extraction, key node identification, and visual explanation output.

[0051] Step 6.1: Attention weight extraction. In the Cross-modal Clue Mining (CCM) module and the Adaptive Attention Aggregation (AAA) module, the attention weights between node pairs have been obtained. Let the final fused multimodal graph be , where V is a node set, representing a modal unit (image region, text semantic fragment, etc.); E is an edge set, representing the association relationship between modalities or within modalities; for node pairs , and its attention weight is expressed as: (26) in, For nodes The feature representation of ; W is a trainable linear transformation matrix; is the weight vector used to calculate the attention score; Represents vector concatenation operation; For nodes The set of neighbors of Indicates that in the process of graph propagation and fusion, the nodes For Node The degree of information contribution.

[0052] Step 6.2: Identification of key nodes and paths. Select key nodes based on their total attention scores to measure their importance in the final decision. , whose total attention weight is defined as: (27) Score the attention Sorting, selecting Top-K nodes as the key node set , these nodes are regarded as information sources that play a key role in model judgment. At the same time, a maximum weight propagation path can be constructed based on the attention matrix to reflect the main information flow between modalities: (28) in, is the attention distribution matrix, each row is the importance score of the current node to all neighbors; is the maximum weight propagation path.

[0053] Step 6.3: Visualize and interpret the output. With path , which produces the following interpretable output: (1) Text part: text segments with higher annotation weights; (2) Image part: Mark the key visual areas through visual weight heatmap (such as Grad-CAM or Attention Heatmap); (3) Graph structure: Display important nodes and edges, and propagate paths based on the maximum weight. Constructing a subgraph , for analysts to understand the decision logic; (4) Contribution ranking chart: displays the contribution value of each mode or node Distribution, enhancing decision-making transparency.

[0054] In summary, this paper effectively improves the performance of multimodal fake news detection through carefully designed modal feature decoupling, cross-modal clue mining and adaptive aggregation mechanism, while also having good interpretability.

[0055] Optionally, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the various processes of the above-mentioned embodiment of the multimodal fake news detection method based on decoupling and cross-modal clue mining are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0056] An embodiment of the present application also provides a readable storage medium having a program or instruction stored thereon. When the program or instruction is executed by a processor, the various processes of the embodiment of the above-mentioned multimodal fake news detection method based on decoupling and cross-modal clue mining are implemented, and the same technical effect can be achieved. To avoid repetition, they are not further described here.

[0057] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0058] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0059] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0060] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A multimodal fake news detection method based on decoupling and cross-modal clue mining, characterized by: The following steps are involved: Step 1: Modal feature extraction: For multimodal news data containing images and text, a pre-trained contrastive learning model is used to synchronously map the multimodal news data of images and text into a unified embedding space to generate aligned modal feature representations; Step 2: Multimodal feature decoupling: The aligned modal feature representations are input into a multimodal decoupling autoencoder. By adversarial similarity constraints and difference constraints, the feature representation of each modality is decomposed into shared features common to the modalities and specific features unique to the modality. Step 3: Cross-modal clue mining: Based on the decoupled shared features and specific features, the cross-modal interaction module is used to model the semantic association between images and texts to mine complementary, reinforcing, and conflicting semantic clues and generate cross-modal interaction features. Step 4: Adaptive attention aggregation: Introduce an adaptive attention aggregation module to dynamically weight the modal feature representation, shared features, specific features, and cross-modal interaction features to generate a fused representation that can highlight key discriminant information; Step 5, false news classification and identification: The fusion representation is input into the classifier, and the model is trained by jointly optimizing the classification loss and the decoupling loss to achieve the authenticity classification of the multimodal news data.

2. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 is characterized in that: In the step 1, it specifically includes: A pre-trained contrastive language-image model is used as the contrastive learning model, and its text encoder and image encoder are used to extract text feature vectors and image feature vectors respectively; The text feature vector and the image feature vector are input into a linear mapping layer to transform them into a unified low-dimensional embedding space to further eliminate the distribution differences between the modalities, and the mapped feature representation is normalized.

3. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 is characterized in that: In step 2, the multimodal decoupling autoencoder is composed of a shared encoder and a modality-specific encoder; The shared encoder is used to extract common shared features from feature representations of different modalities; The modality-specific encoder is used to learn unique specific features from the feature representation of each modality; By introducing a reconstruction decoder, the original input features are reconstructed intra-modally and cross-modally to ensure the information integrity of the decoupled features.

4. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 3 is characterized in that: In step 2, the adversarial similarity constraint is implemented by introducing a discriminator, which is used to distinguish the modal sources of shared features, and drives the shared encoder to generate a common representation that is consistent across modalities and cannot distinguish the source through adversarial training; The difference constraint is implemented by calculating the cosine similarity between the shared features and the specific features to ensure the semantic difference between the shared features and the specific features.

5. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 is characterized in that: In step 3, the cross-modal clue mining specifically includes: For shared features, a dual-stream gated attention mechanism is used to achieve text-guided image shared feature enhancement and image-guided text shared feature enhancement; Aiming at specific features, a bidirectional interactive network is used to achieve text-guided image-specific semantic completion and image-guided text-specific semantic completion. The enhanced shared features and the completed specific features are spliced ​​and fused to form cross-modal interaction features that comprehensively reflect the semantic consistency and conflict between images and texts.

6. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 is characterized in that: In step 4, the adaptive attention aggregation module specifically includes: A multi-head attention mechanism is used to perform preliminary aggregation of the concatenated modal features and interaction features; A modal confidence assessment network is used to dynamically evaluate the contribution of each modality and interaction feature in the current discrimination task and generate corresponding confidence weights; The confidence weights are combined to perform weighted summation on each feature to generate the final fusion representation.

7. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 or 5, characterized in that: In step 5, the classifier is a multi-layer perceptron, and a cross-entropy loss function is used as the classification loss; the goal of the model training is the weighted sum of the cross-entropy loss and the decoupling loss of the multimodal decoupling autoencoder, and the model is end-to-end jointly optimized through a back-propagation algorithm.

8. The multimodal fake news detection method based on decoupling and cross-modal clue mining according to claim 1 is characterized in that: The method also includes step 6, interpretability analysis: in order to visually explain the decision-making process of the model, an interpretability module based on the graph attention mechanism is introduced; by analyzing the attention weight distribution in the adaptive attention aggregation module, the key image areas, key text fragments and interaction paths between modalities that contribute most to the classification results are identified and output, providing a transparent decision-making basis for content review and public opinion analysis.

9. An electronic device, characterized in that: The system comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the system implements the steps of a multimodal fake news detection method based on decoupling and cross-modal clue mining as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction. When the program or instruction is executed by the processor, the steps of the multimodal fake news detection method based on decoupling and cross-modal clue mining as described in any one of claims 1 to 8 are implemented.

Images