A multi-modal fake news detection method based on hierarchical logic predicates and bidirectional mamba

Abstract

The application discloses a kind of multi-modal false news detection method based on hierarchical logic predicate and two-way Mamba, belong to artificial intelligence and data processing technical field.The application is respectively to news text and supporting image feature coding and fusion, constructs relationship class, fraud mechanism class and image-text pairing class basic logic predicate, and generates derivative predicate by microfuzzy logic;Each predicate is mapped as vector token and is divided into relationship group, mechanism group, pairing group and derivative group, and reasoning is carried out in group by using parameter-shared local two-way Mamba, and complete enhanced token sequence of each group is retained;After the four groups of enhanced sequences are spliced, global two-way Mamba is input to carry out cross-group reasoning, and the true or false news result is output by two-class main head, when having corresponding fine-grained mechanism label, six-class fine-grained auxiliary head is set in training stage, and explicit logic expression and hierarchical joint reasoning of multi-modal evidence are realized.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and data processing technology, specifically relating to a multimodal fake news detection method based on hierarchical logical predicates and bidirectional Mamba. Background Technology

[0002] With the rapid development of social media platforms, news content is increasingly being disseminated in the form of both text and images. Fake news publishers can mislead readers not only by fabricating textual facts, but also by using repurposed old images, misleading links, image manipulation, impersonating sources, using satirical expressions, or misleading combinations of text and images to enhance the deceptiveness of their content. Therefore, relying solely on textual semantics or single image content to determine authenticity is insufficient to fully encompass the complex deceptive patterns inherent in multimodal fake news.

[0003] Existing multimodal fake news detection methods typically employ text encoders and visual encoders to extract continuous features, then achieve multimodal fusion through methods such as concatenation, attention, gating, or contrastive learning. While these methods can utilize textual and image information, the final determination of truth or falsehood often occurs directly on the high-dimensional continuous representation. They lack intermediate inference variables with clear semantic meanings, such as "credible," "contradictory," "uncertain," "image-text matching," "erroneous connections," and "tampering," making it difficult for the model to explicitly express the logical support, conflicts, and complex deception relationships between different pieces of evidence.

[0004] Some methods attempt to introduce neural symbolic variables or artificial rules, but if only a small number of discrete variables are set, it is difficult to simultaneously cover relational states, deception mechanisms, image-text pairing, and high-order complex logic. If all logical variables are input into the same inferencer at once, their natural semantic groupings are easily overlooked. Relational, mechanism, pairing, and derived predicates have different internal dependency structures, so it is more suitable to first perform intra-group context modeling, and then retain the enhanced token-level evidence within the group to enter cross-group global inference.

[0005] Furthermore, logical predicate sequences do not possess a natural unidirectional temporal order. If only a unidirectional scan is used, the preceding predicates cannot receive the state information of the following predicates, making it easy for the manually arranged order to have a directional impact on the reasoning results. Bidirectional state-space scanning can selectively propagate states in both the original and reversed order, and then align and fuse the information from both directions, providing a bidirectional context for structured predicate sequences.

[0006] Mamba, based on a selective state-space model, can dynamically adjust state updates according to the current input, achieving sequence state propagation without constructing pairwise fully connected attention matrices. However, if Mamba is only used as a low-level text or visual encoder, it is still difficult to directly reflect the explicit logical relationships in fake news detection. Therefore, it is necessary to first convert the image-text fusion features into basic predicates and derived predicates with clear semantics, and then use local and global bidirectional Mamba for hierarchical logical reasoning.

[0007] In summary, existing multimodal fake news detection methods still suffer from problems such as insufficient explicit logical evidence expression, lack of grouping modeling of logical variables of different semantic types, and difficulty in fully utilizing bidirectional dependencies between predicates in unidirectional or flat reasoning. It is necessary to establish a hierarchical logical reasoning mechanism that is differentiable, grouped, and has cross-group joint modeling capabilities on top of multimodal features. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a multimodal fake news detection method based on hierarchical logical predicates and bidirectional Mamba. The method first encodes the news text and accompanying images separately, and then concatenates the two modal features to form a unified multimodal feature. Next, a predicate generator constructs basic logical predicates for relational, deception mechanism, and image-text pairing categories, and then uses differentiable fuzzy logic to construct derived logical predicates. Subsequently, each predicate is mapped to a token and divided into four logical groups. Intra-group bidirectional reasoning is performed on each group using parameter-shared local bidirectional Mamba, resulting in four sets of locally enhanced predicate sequences. These four enhanced sequences are concatenated along the token dimension to form a global logical sequence, which is then used for cross-group logical reasoning via global bidirectional Mamba to obtain a structured global reasoning state. Finally, a binary classification master head outputs the true / false news results, and a six-classification auxiliary head and logical consistency constraints can be set during the training phase.

[0009] This invention is achieved through the following technical solution: a multimodal fake news detection method based on hierarchical logical predicates and bidirectional Mamba, comprising the following steps:

[0010] Step 1, Multimodal Feature Encoding and Fusion: The input news text and accompanying image are processed by pre-trained text encoder and pre-trained image encoder respectively to extract text features and image features. The features are then concatenated or equivalently fused in the feature dimension to form multimodal fused features.

[0011] The specific method is as follows:

[0012] Input the news text x into the pre-trained text encoder E t The news image I is input into the pre-trained image encoder E. i Global pooling or equivalent feature convergence is performed on the encoding results respectively to obtain the text features F.t and image features F i :

[0013]

[0014] Where, d t and d i d represents the text feature dimension and image feature dimension, respectively; the text encoder uses a pre-trained language model capable of outputting contextual semantic representations, and the image encoder uses a pre-trained visual network capable of outputting visual semantic representations; the text encoder uses MacBERT, and the image encoder uses SwinTransformer; PoolText and PoolImage represent the global pooling or equivalent feature convergence operators of the text encoder output and the image encoder output, respectively; the equivalent feature convergence refers to the differentiable convergence operation that converts the sequence features output by the text encoder or the spatial features output by the image encoder into a fixed-dimensional feature vector; F Let x represent the dimension of the multimodal feature F obtained by concatenating text features and image features; let I represent the input news text; let E represent the input news image corresponding to the news text; and let E represent the dimension of the multimodal feature F obtained by concatenating text features and image features. t and E i F represents the pre-trained text encoder and the pre-trained image encoder, respectively. t and F i These represent the text feature vector and image feature vector obtained by the corresponding convergence operator, respectively.

[0015] Text features F t With image features F i By concatenating the features along the feature dimension, we obtain the unified multimodal feature F:

[0016]

[0017] The multimodal features F are input into a shared predicate feature mapping network to form a predicate generation representation that can be used by multiple predicate output heads:

[0018]

[0019] Among them, h p This represents the predicate generation representation shared by multiple basic predicate output heads; φ p This represents a predicate generator consisting of one or more fully connected layers, normalization layers, nonlinear activation layers, or combinations thereof; the predicate generation representation simultaneously includes textual semantic information, visual semantic information, and the correspondence between the two in the joint feature space; d p This represents the dimension of the shared predicate generation representation.

[0020] Step 2, Construction of basic logical predicates: Input the multimodal fusion features into a learnable predicate generator to generate multiple sets of basic logical predicates, including relational predicates, deception mechanism predicates, and image-text pairing predicates;

[0021] In step 2, the predicate generator forms a predicate generation representation h based on multimodal fusion features, which can be shared by multiple basic predicate output heads. p The dimension of the predicate generation representation is d. p The predicate generator connects to three basic predicate output heads, obtaining the basic predicate probabilities for relational, deception mechanism, and image-text pairing classes, respectively.

[0022]

[0023]

[0024]

[0025] Among them, W R W M W P These represent the weight parameters of the predicate output heads for relational, deception mechanism, and image-text pairing classes, respectively. R b M b P For the corresponding bias parameter, p R p M p P These are the corresponding predicate probability vectors; relational predicates include four predicates: trusted, weak, contradiction, and uncertain; deception mechanism predicates include six predicates: true, satire, false_connection, imposter, manipulated, and misleading; and image-text pairing predicates include two predicates: pair_match and pair_mismatch, thus forming twelve basic logical predicates;

[0026] Each predicate output head shares the predicate generation representation obtained from the same multimodal fusion feature, while the output layer maintains independent parameters; the relational predicate output head, the deception mechanism predicate output head, and the image-text pairing predicate output head output 4-dimensional, 6-dimensional, and 2-dimensional predicate probability vectors p, respectively. R p M and p P Accordingly, W R W M and W P The matrix sizes are 4×d p 6×d p and 2×d p b Rb M and b P The dimensions are 4, 6, and 2 respectively; softmax represents the normalization exponential function, used to convert the unnormalized scores of each output head into corresponding predicate probability vectors. Relational predicate probability vector p R The components are p in order. trusted p weak p contradiction and p uncertain Deception mechanism class predicate probability vector p M The components are p in order. true p satire p false_connection p imposter p manipulated and p misleading Image-text pairing class predicate probability vector p P The components are p in order. pair_match and p pair_mismatch All of the above probability components are located in [0,1], and the sum of all components of the same predicate probability vector is 1.

[0027] Step 3: Derived logic predicate construction: According to the preset logical combination relationship, the basic logical predicates are combined using differentiable fuzzy logic operators to generate derived logic predicates; wherein, some derived logic predicates are obtained by directly combining multiple basic logical predicates, and some derived logic predicates are obtained by further combining basic logical predicates with the generated derived logic predicates;

[0028] In step 3, differentiable fuzzy logic is used to combine the basic predicates and the derived logic predicates constructed from the basic predicates step by step; fuzzy AND and fuzzy OR are defined as follows:

[0029]

[0030]

[0031] Among them, a u ∈[0,1] represents the truth value of the u-th predicate participating in the combination, and n represents the number of predicates participating in the current fuzzy logic combination; A f and O f These represent the fuzzy AND operator and the fuzzy OR operator, respectively, with the subscript f indicating the identifier subscript of the fuzzy logic operator. Based on the above operators, this embodiment constructs eight derived logic predicates: relation conflict, arbitrary spurious mechanism, truth support, semantic deception, cross-modal deception, contradictory spurious, paired truth support, and paired conflict spurious. These eight derived logic predicates are constructed according to the following differentiable fuzzy logic relations:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Where, p RC p FA p TS p SD p CMD p CF p PTS and p PCF These represent relational conflict, arbitrary false mechanism, true support, semantic deception, cross-modal deception, contradictory false, paired true support, and paired conflict false, respectively; the underlying predicate truth value p participating in the above derivational combinations trusted p weak p contradiction p uncertain p true p satire p false_connection p imposter p manipulated p misleading p pair_match p pair_mismatch These represent the truth values ​​of the twelve basic logical predicates, all of which are located in the range [0,1]. FA and p TS In subsequent combinatorial expressions, the derived logic predicates that have been constructed in advance are represented and can continue to be used as inputs to differentiable fuzzy logic operators in successive combinations.

[0041] Step 4, Predicate Tokenization and Logical Grouping: Map the scalar values ​​of basic logical predicates and derived logical predicates to vector tokens, and divide them into relation groups, mechanism groups, pairing groups and derived groups according to the semantic function of the predicates;

[0042] In step 4, the truth value p of the j-th logical predicate is... j Mapped to d by learnable value projection tok Dimensional token space; will p j Its inverse truth value 1-p jBoth are used as input to the value projection, and predicate type embeddings and logical group embeddings are superimposed:

[0043]

[0044] Where, φ v Represents a predicate-valued projection network; e j type Let e ​​represent the semantic identity embedding of the j-th predicate. g(j) group This represents the group embedding of the logical group to which the j-th predicate belongs; both the predicate semantic identity embedding and the logical group embedding are trainable parameters with dimension d. tok These are used to identify the specific semantic identity of the predicate and its corresponding logical group. The initial values ​​of the predicate semantic identity embedding and the logical group embedding can be initialized randomly or by an equivalent trainable method, and are updated together with the parameters of the predicate value projection network during model training.

[0045] The twelve basic predicates and eight derived predicates are organized into the following four groups: where j represents the predicate index, g(j) represents the logical group to which the j-th predicate belongs, and d tok This indicates a unified predicate token that hides a dimension; e j G represents the vector token corresponding to the j-th predicate; R G M G P and G D These represent predicate token sequences for relation groups, mechanism groups, pairing groups, and derivation groups, respectively; p j ∈[0,1];φ v By p j With 1-p j The constructed two-dimensional numerical vector serves as the input, and d is the output. tok Dimensional vector.

[0046]

[0047]

[0048]

[0049]

[0050] Among them, e trusted e weak e contradiction e uncertain e true e satire e false_connection e imposter e manipulated e misleadinge pair_match and e pair_mismatch Each represents a vector token corresponding to one of the twelve basic logical predicates; e RC e FA e TS e SD e CMD e CF e PTS and e PCF G represents the vector tokens corresponding to the eight derived logical predicates: relational conflict, arbitrary spurious mechanism, true support, semantic deception, cross-modal deception, contradictory spurious, paired true support, and paired conflict spurious; R G M G P and G D Let represent the predicate token sequences of the relation group, mechanism group, pairing group, and derivation group, respectively. The number of tokens in each group is 4, 6, 2, and 8, respectively, and the hidden dimension of each predicate token is d. tok .

[0051] Step 5: Shared Local Bidirectional Mamba In-Group Inference: Input the predicate token sequences from the relation group, mechanism group, pairing group, and derivation group into the parameter-shared local bidirectional Mamba; for the predicate token sequence of each logic group, perform forward selective state space scan and reverse selective state space scan respectively, and fuse the outputs of the two scan directions to obtain a locally enhanced predicate sequence with the same length as the input sequence of the logic group.

[0052] In step 5, for any logical group g, intra-group inference is performed using local bidirectional Mamba with shared parameters among the four logical groups:

[0053]

[0054]

[0055] Among them, M L This represents a local Mamba that shares parameters among four logical groups; Rev(·) represents reversing the sequence order. g H represents the input token sequence of the g-th logic group. g → and H g ← These represent the forward scan result and the reverse scan result after restoring the original order, respectively; the forward output and the reverse output after restoring the original order are fused, residual connected, and normalized to obtain the locally enhanced predicate sequence C. g And keep the number of tokens in this logical group unchanged; W L and bL respectively denote the weight matrix and bias vector for bidirectional state fusion, LN denotes layer normalization, L g denotes the number of tokens in the g-th logical group. W L and its transpose matrix W L T have dimensions of d tok ×2d tok and 2d tok ×d tok respectively; the bidirectional concatenated state is right-multiplied by W in the formula L T to map back to d on a token-by-token basis tok dimensions; b L is a d tok -dimensional bias vector, which is broadcast and added according to the token position; G g , H g → and H g ← all have dimensions of L g ×d tok ; the semicolon ";" in the formula denotes concatenating the states of the two directions along the feature dimension. g∈{R,M,P,D}, where R, M, P, and D respectively denote the relation group, the mechanism group, the pairing group, and the derivation group, and the corresponding locally enhanced predicate sequences C g are respectively denoted as C R , C M , C P and C D .

[0056]

[0057] Herein, C g denotes the predicate token sequence of the g-th logical group after enhancement by the shared local bidirectional Mamba, and the locally enhanced predicate sequences corresponding to the relation group, mechanism group, pairing group, and derivation group are respectively denoted as C R , C M , C P and C D ; L g denotes the number of tokens in the g-th logical group, and L R , L M , L P and L D are 4, 6, 2 and 8 respectively; d tok denotes the hidden dimension of each predicate token. Therefore, the shared local bidirectional Mamba only enhances the intra-group context representation, and does not change the number of tokens in the corresponding logical group and the hidden dimension of a single token.

[0058] Step 6: Global Bidirectional Mamba Cross-Group Reasoning: The local enhanced predicate sequences corresponding to the relation group, mechanism group, pairing group, and derivation group are concatenated according to the preset logical group order to form a global logical sequence; the global bidirectional Mamba is used to perform forward and reverse selective state space scanning on the global logical sequence respectively, and the scanning results of the two directions are fused to obtain a structured global reasoning state; the structured global reasoning state is feature-aggregated to obtain a global logical representation;

[0059] In step 6, the locally enhanced predicate sequences C of the relation group, mechanism group, pairing group, and derivation group are... R C M C P and C D Concatenate them sequentially to form a global logical sequence:

[0060]

[0061] Perform forward and reverse scans on the global logical sequence using a global bidirectional Mamba independent of the local inferencer:

[0062]

[0063]

[0064]

[0065] Among them, C G L represents the global logical sequence obtained by concatenating four sets of locally enhanced predicate sequences. G M represents the length of its token. G Represents global Mamba, Rev(·) reverses the sequence order, U → and U ← W represents the global forward scan result and the reverse scan result after restoring the original order, respectively. G and b G Represent the weight matrix and bias vector for global bidirectional state fusion, respectively; LN denotes layer normalization; H G h represents the cross-group reasoning state that maintains the global sequence structure. G Indicates H G The compact global logic representation obtained through feature aggregation; PoolLogic represents the structured global reasoning state H. GExecute operators for learnable pooling, attention pooling, mean pooling or equivalent feature aggregation, all of said aggregation methods are differentiable operations, and their output dimensions are consistent with the hidden dimension of a single predicate token, so as to ensure that in the training phase, gradients can be back-propagated from the global logical representation to the preceding inference module; wherein, the equivalent feature aggregation refers to a sequence aggregation operation that takes a sequence state as input, outputs a single fixed-dimension global representation and remains differentiable. W G and its transposed matrix W G T have dimensions of d respectively tok ×2d tok and 2d tok ×d tok ; the bidirectionally concatenated global state is right-multiplied by W in the formula G T , mapping back to d per token tok dimension; b G is a d tok -dimensional bias vector, which is broadcast and added by token position; H G retains L G tokens, and the dimension of each token is d tok , the dimension of h G is d tok ; in the formula, the semicolon ";" denotes concatenating the global states of two directions along the feature dimension; Concat denotes concatenating along the token dimension in the preset order of the relation group, mechanism group, pairing group and derivation group; C G , U → , U ← and H G all have dimensions of L G ×d tok , and L G =L R +L M +L P +L D . Here, d tok denotes the unified hidden dimension of a single predicate token; L R , L M , L P and L D respectively denote the number of tokens of the relation group, mechanism group, pairing group and derivation group, with the values being 4, 6, 2 and 8 in sequence; L G denotes the total number of tokens in the global logical sequence.

[0066] Step 7, Model Training and Fake News Detection: Input the global logical representation into the main classification head to obtain the prediction results of real news and fake news; when there are corresponding fine-grained mechanism labels, set a fine-grained auxiliary classification head during the training phase and optimize the model parameters through joint loss; during the detection phase, use the output of the main classification head as the final real and fake news judgment result.

[0067] In step 7, the binary classification master head and the six-class classification auxiliary head in the training phase are respectively represented by global logic h. G For input:

[0068]

[0069]

[0070] Among them, h G This represents the global logical representation, with dimension d. tok ;d tok This indicates a unified predicate token that hides dimensions; bin and l aux W represents the unnormalized classification scores of the binary primary head and the six-class auxiliary head, with dimensions of 2 and 6, respectively; bin and W aux and b bin and b aux These represent the weight matrix and bias vector of the corresponding classification head, respectively.

[0071] Among them, W bin The size is 2×d tok b bin The dimension of W is 2; aux The size is 6×d tok b aux The dimension is 6. ŷ and q̂ represent the binary and six-class prediction probability vectors after softmax normalization, respectively; softmax represents the normalization exponential function, with each probability component located in [0,1], and the sum of all components of the same probability vector is 1.

[0072] The unnormalized scores and probability vectors for binary classification are arranged in the order of "real news, fake news"; the unnormalized scores and probability vectors for six-class classification are arranged in the order of true, satisfied, false connection, imposter, manipulated, and misleading. The binary classification master head is retained during both the training and detection phases; the six-class auxiliary head only participates in optimization during the training phase when it has corresponding fine-grained supervised labels, and does not participate in the final class determination during the detection phase. The final detection result is determined by the output of the binary classification master head.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] (1) This invention splices or equivalently fuses text features and image features in the feature dimension, and then the learnable predicate generator explicitly converts the obtained multimodal fusion features into basic logical predicates with clear semantic meanings, forming a structured detection link of "multimodal features - logical predicates - hierarchical reasoning - classification decision".

[0075] (2) This invention simultaneously models different basic logical spaces such as relational states, deception mechanisms and image-text pairing, and uses differentiable logic to generate higher-order derived predicates, enabling the model to explicitly express various evidentiary relationships such as credible support, contradiction, pairing conflict and compound deception.

[0076] (3) The present invention uses differentiable fuzzy AND, fuzzy OR and their equivalent logic operators to construct the basic predicate into derived logic evidence, so that the preset logical combination relationship can be optimized end-to-end with the neural network.

[0077] (4) This invention uses parameter-shared local bidirectional Mamba to perform intra-group bidirectional reasoning within the four logical groups and retains the complete local enhanced token sequence of each group; in the global stage, the four enhanced sequences are spliced ​​together and then cross-group bidirectional reasoning is performed by global bidirectional Mamba, which can jointly model high-order combined evidence and cross-group deception patterns between relation groups, mechanism groups, pairing groups and derived groups, while avoiding the loss of fine-grained evidence caused by premature compression in the local stage.

[0078] (5) Both the local and global stages employ forward and reverse selective scanning, enabling each predicate to simultaneously receive the state information of its preceding and following semantic positions, thereby reducing the one-way propagation bias caused by the fixed predicate arrangement order.

[0079] (6) The present invention sets up a binary classification master head and an optional fine-grained auxiliary head. The auxiliary head is only used to provide supervision during the training phase and can be removed during the deployment phase, thereby improving the discriminativeness of the global logical representation by using fine-grained labels while keeping the final detection path simple.

[0080] (7) The core reasoning of this invention only relies on the input image and text, pre-trained perceptual encoder, explicit logical predicate and bidirectional state space inferencer. It does not require calling external large language models to generate background knowledge or reasoning text, which is convenient to implement and reproduce in a fixed model version and offline environment. Attached Figure Description

[0081] Figure 1 A flowchart illustrating the overall process of a multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba;

[0082] Figure 2This is a diagram illustrating the overall structure of the hierarchical logic predicate reasoning model of this invention.

[0083] Figure 3 A schematic diagram illustrating the encoding and fusion of multimodal features and the construction of twelve basic logical predicates;

[0084] Figure 4 A schematic diagram illustrating the fuzzy logic combination relationship between twelve basic logic predicates and eight derived predicates;

[0085] Figure 5 A schematic diagram of the four-logic-group intra-group sequence reasoning structure for a shared local bidirectional Mamba;

[0086] Figure 6 This is a schematic diagram of the structure of four sets of locally enhanced sequences spliced ​​together, global bidirectional Mamba, a two-classification master head and a six-classification auxiliary head. Detailed Implementation

[0087] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the implementation methods of the present invention, and not all of them. Other equivalent implementation methods obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the protection scope of the present invention.

[0088] The overall process and hierarchical logical predicate reasoning structure of this embodiment are as follows: Figures 1 to 6 As shown, the technical solution of the present invention will be described below in conjunction with each processing step.

[0089] 1. Multimodal input, feature encoding and fusion.

[0090] This embodiment is designed for multimodal news samples that include news text and accompanying images. The training samples contain at least text, images, and binary labels for true / false; on data with fine-grained labeling mechanisms, auxiliary labels such as true, satire, false_connection, imposter, manipulated, and misleading may also be included.

[0091] In a preferred embodiment, the text encoder uses MacBERT, and the image encoder uses SwinTransformer. The context representation output by the text encoder is converged into text features F using CLS, mean pooling, or an equivalent method. t The visual representation output by the image encoder is pooled into image features F through global pooling or an equivalent method. i Here, CLS represents the global semantic representation corresponding to the classification labels in the pre-trained language model.

[0092] The text encoder and image encoder can be frozen, partially unfrozen, or adapted to the task using hierarchical learning rates; the feature dimensions can be unified through linear projection. In this embodiment, the convergent features of the two modalities are directly concatenated along the feature dimension, and the input to the predicate generator can be formed without requiring an additional cross-modal attention module.

[0093] Text features, image features, and their splicing relationships can be represented as:

[0094]

[0095]

[0096] The concatenated multimodal features F are input to a predicate generator MLP. This generator can be configured with a shared hidden representation, and then output three heads: connection relations, mechanisms, and pairings; thus, it shares the image-text fusion information while preserving the output semantics of different predicate spaces. Here, MLP stands for Multilayer Perceptron; F... t and F i Let d represent the text feature vector and the image feature vector, respectively. t and d i They represent the feature dimensions of the two components respectively; F represents the concatenated multimodal features, and d F =d t +d i This indicates its feature dimension.

[0097] 2. Twelve basic logical predicates.

[0098] The multimodal fusion feature F, or its shared predicate representation obtained through MLP, is input into three basic predicate output heads to obtain a 4-dimensional relational predicate probability vector, a 6-dimensional deception mechanism predicate probability vector, and a 2-dimensional image-text pairing predicate probability vector, respectively. The weight parameters of the three output heads are denoted as W. R W M W P The bias parameters are denoted as b. R b M b P .

[0099] Relational predicates include trusted, weak, contradiction, and uncertain. Trusted indicates strong credible support, weak indicates a weak relationship or insufficient support, contradiction indicates a significant contradiction, and uncertain indicates insufficient evidence or an uncertain relationship.

[0100] Predicates for deception mechanisms include true, satire, false_connection, imposter, manipulated, and misleading, used to describe fine-grained news mechanisms such as truthfulness, irony, false connections, impersonation, tampering, and misleading.

[0101] Image-text pairing predicates include `pair_match` and `pair_mismatch`, which directly characterize whether the text and the accompanying image constitute a mutually supportive semantic pair. All of these predicate probabilities are generated from the same multimodal fusion feature, but each output head has independent output layer parameters.

[0102] In obtaining the basic predicate generator, if the training data contains supervision information corresponding to relational states, deception mechanisms, or image-text pairing relationships, the corresponding predicate output heads can be pre-trained or initialized accordingly. If some predicates lack direct supervision, a pre-initialized predicate generator can be used, or the generator can be optimized by a binary classification main task, an available six-classification auxiliary task, and logical consistency constraints during the training phase that allows updating the predicate generator. Relational predicates describe the overall support, conflict, or uncertainty state of joint image-text evidence, while image-text pairing predicates describe the direct semantic correspondence between the text and the current image; the two serve different logical functions.

[0103] 3. Eight derived logic predicates.

[0104] Starting with twelve basic predicates, differentiable fuzzy logic composition is performed, and the constructed derived predicates are allowed to participate in subsequent hierarchical combinations, forming eight higher-order derived predicates. Fuzzy AND uses a product form, and fuzzy OR uses a complement product form, enabling the logic composition itself to participate in backpropagation.

[0105] `relation_conflict` integrates `weak`, `contradiction`, and `uncertain`; `fake_any` aggregates five types of non-real mechanisms; `true_support` combines `trusted` and `true`; `semantic_deception` aggregates `false_connection`, `imposter`, and `misleading`; `cross_modal_deception` aggregates `pair_mismatch`, `false_connection`, and `manipulated` related to text-image consistency anomalies; `contradiction_fake` combines `contradiction` and `fake_any`; `pair_true_support` combines `pair_match` and `true_support`; `pair_conflict_fake` combines `pair_mismatch` and `fake_any`. In this embodiment, `relation_conflict` adopts a broad meaning of relational anomalies, not just the narrow meaning of `contradiction`.

[0106] in, Figure 4 The "+" sign in the diagram indicates that multiple predicates participate in the same fuzzy logic combination, and does not indicate arithmetic addition between the truth values ​​of the predicates; the specific combination relationship between multiple predicates is determined by the corresponding fuzzy AND operator or fuzzy OR operator in the diagram.

[0107] Through the derived predicates mentioned above, some higher-order relations with clear prior meanings have been explicitly constructed before entering Mamba inference; at the same time, all logical operators are differentiable, so the final classification loss can influence the basic predicate generator in reverse through the derived predicates.

[0108] 4. The predicate token indicates a four-logic group structure.

[0109] Each predicate scalar p j Learnable values ​​are projected onto a unified hidden dimension. In a preferred embodiment, p... j With 1-p j Simultaneously, the input value is projected onto the network, and predicate type embedding and logic group embedding are superimposed to simultaneously represent "predicate validity, anti-truth value, specific semantic identity, and belonging logic group".

[0110] Type embedding and group embedding are not the only implementation methods. Independent projection, fixed semantic position encoding, or lightweight group adapters can also be used to achieve equivalent predicate identity differentiation.

[0111] In this embodiment, the lengths of the relation group, mechanism group, pairing group, and derived group are 4, 6, 2, and 8, respectively; a fixed semantic order is used within the group to form repeatable inputs, and bidirectional Mamba is used to reduce the impact of this artificial order on unidirectional state propagation.

[0112] 5. Shared local bidirectional Mamba group inference.

[0113] The relation group, mechanism group, pairing group, and derivation group each receive a shared local bidirectional Mamba sequence with the same parameters. The forward branch scans along the fixed semantic order within the group, while the reverse branch scans the reversed sequence within the group and restores the original order. The states from both directions are linearly fused, residually connected, and layered normalized to obtain the locally enhanced predicate sequence C. R C M C P and C D .

[0114] The shared local bidirectional Mamba learns a general intra-group evidence state propagation mechanism. The semantic differences between the four groups of relation, mechanism, pairing and derivation are distinguished by the predicate identity and group identity of the input token.

[0115] This embodiment does not immediately compress each group into a single summary after local inference, but instead retains the complete enhanced token sequence for each group. Therefore, the relation group, mechanism group, pairing group, and derived group retain L respectively. R L M L P and L D Each local enhanced state provides fine-grained input for the global stage. The local bidirectional state fusion weight matrix W... L The size is d tok ×2d tok Bias vector b L The dimension is d tok ;d tok The unified predicate token represents a hidden dimension.

[0116] 6. Global bidirectional Mamba cross-group logical reasoning.

[0117] The locally enhanced predicate sequences C corresponding to the relation group, mechanism group, pairing group, and derivation group are... R C M C P and C D By directly concatenating the logical groups according to the preset logical order, we obtain the global logical sequence C. G Its sequence length is L G =L R +L M +L P +L DThe parameters of the global bidirectional Mamba and the shared local bidirectional Mamba are independent, and forward and reverse selective scans are performed on the global logical sequence.

[0118] For example, when the relationship group shows a strong "contradictory" state, the mechanism group shows "tampering" or "erroneous connection", the pairing group shows "image and text mismatch", and the "cross-modal deception" in the derived group is also high, global reasoning can unite evidence from different logical spaces at the token level; conversely, "credible", "real", "image and text matching" and "real support" can form a mutually supportive chain of real evidence.

[0119] The input to the global bidirectional Mamba is not the original image patch or the original text token, but a logical token that has undergone explicit semantic abstraction and local context enhancement. The bidirectional global states are fused to form a structured global reasoning state H. G And a compact global logical representation h is obtained through learnable pooling. G Among them, C G L represents the global logical sequence concatenated from four sets of local enhancement sequences. G M represents its length. G Represents global Mamba, U → and U ← W represents the global forward and reverse scan results, respectively. G and b G H is the global bidirectional state fusion parameter. G Represents the structured global reasoning state, PoolLogic represents the global logic feature aggregation operator, h G This represents the converged global logical representation. `Concat` indicates concatenation along the token dimension according to a preset order of relation group, mechanism group, pairing group, and derived group. The global bidirectional state fusion weight matrix W... G The size is d tok ×2d tok Bias vector b G The dimension is d tok .

[0120] 7. Main classification head, training auxiliary head, and detection.

[0121] Global logical representation h G Input a binary classification master head, and output the probabilities of true and false categories. For data with six-class fine-grained labeling, an additional six-class auxiliary head can be set; the auxiliary head only participates in loss calculation during training and can be removed during deployment and detection.

[0122] During training, the aforementioned joint loss is used: for the primary binary classification task, a smoothed cross-entropy with a label weight of 0.01 is applied, along with a softmacro-F1 loss with a weight of 0.30; for the secondary six-class classification task, a smoothed cross-entropy with class weights and a label weight of 0.01 is applied, with a weight of 0.20; the weights for primary-secondary binary classification consistency, explicit logical rule consistency, and predicate parameter anchoring loss are 0.06, 0.05, and 0.02, respectively. The six-class classification class weights are calculated based on the square root inverse frequency of each class in the training set and normalized to a mean of 1. When the training samples do not have six-class fine-grained mechanism labels, the secondary six-class classification cross-entropy loss and the primary-secondary binary classification consistency loss are not enabled; the predicate parameter anchoring loss is only enabled during the optional predicate fine-tuning stage.

[0123] In one specific embodiment, the predicate token hidden dimension is set to 256, the Mamba state dimension is set to 16, the local convolution width is set to 4, and the expand coefficient is set to 2, where the expand coefficient represents the expansion factor of the internal expansion channel dimension of the Mamba module relative to the input hidden dimension; Dropout is set to 0.10; the training batch size is 4, the gradient accumulation step is 2, and the validation batch size is 8. The optimizer uses AdamW, with two momentum parameters of 0.9 and 0.98, a numerical stability term of 1e-8, a weight decay of 1e-2, a gradient clipping threshold of 1.0, and a learning rate using pre-warmed cosine decay. When using a pre-trained or pre-initialized predicate generator, the first stage can freeze the predicate generator and train the hierarchical inferencer and classification head with a learning rate of 5e-5, a warm-up ratio of 0.05, and one training epoch. The second stage can optionally use approximately 15% of the training batches for predicate polishing with a learning rate of 5e-6, a warm-up ratio of 0.02, and one training epoch, using a snapshot of the predicate generation parameters before entering this stage as an anchor point. If the second stage's accuracy and the mean of the macro-F1 score on the validation set are less than 1.5e-4 higher than the first stage, the first stage model is retained. Mamba's internal one-dimensional local convolutions employ padding and pruning methods that preserve sequence length; therefore, when the convolution kernel width is greater than the length of individual logic groups, the number of output tokens remains consistent with the input sequence.

[0124] Model selection and hyperparameter determination should be based on the training and validation sets; the test set is only used for performance evaluation after the model is frozen, and test data should be avoided from participating in model structure selection.

[0125] 8. Testing and Deployment Instructions.

[0126] During the detection phase, the input image and text news to be detected undergoes text / image encoding, feature concatenation, basic predicate generation, derived predicate calculation, four sets of tokenization, shared local bidirectional Mamba, four sets of local enhanced sequence concatenation, global bidirectional Mamba, and a binary classification master head to obtain the probabilities of real and fake news.

[0127] During the detection phase, only the binary classification master head output is used for final deployment and discrimination, and the highest probability decision is executed based on the master classification master head output, thereby maintaining a single-model, end-to-end, structured logical reasoning detection path.

[0128] 9. Other optional implementation methods.

[0129] The pre-trained text encoder of this invention can employ MacBERT, BERT, RoBERTa, or other models capable of outputting contextual text representations; the pre-trained image encoder can employ SwinTransformer, ViT, ConvNeXt, ResNet, or other models capable of outputting visual semantic features. In the Chinese image-text news implementation, a combination of MacBERT and SwinTransformer is preferred.

[0130] The number and names of basic predicates can be adjusted according to the dataset labeling system, but should at least include several explicit logical variables describing multimodal relationships, authenticity or deception mechanisms, and image-text pairing relationships; derived predicates can be constructed using differentiable fuzzy AND, fuzzy OR, fuzzy NOT, or their equivalent t-norm / t-conorm.

[0131] Shared local bidirectional Mamba can share all core parameters among logical groups, or set up lightweight group adapters based on shared core parameters; global bidirectional Mamba receives a global logical sequence formed by splicing local enhancement sequences from each group, and can also add group boundary identifiers or global decision tokens without changing the hierarchical concept.

[0132] The auxiliary classification head is not a necessary component of the deployment phase; on data lacking six fine-grained labels, training can be completed using only the binary classification master task and available logical consistency or predicate stability constraints.

[0133] For tasks such as multi-category fake news detection, event type identification, or other text and image credibility analysis, the main classification head can be extended to multi-category output. As long as the technical links of generating explicit logical predicates, logical grouping, shared local bidirectional state space reasoning, and global bidirectional state space reasoning are still used, they are all equivalent implementations of the concept of this invention.

Claims

1. A multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba, characterized in that, Includes the following steps: Step 1, Multimodal Feature Encoding and Fusion: The input news text and accompanying image are processed by pre-trained text encoder and pre-trained image encoder respectively to extract text features and image features. The features are then concatenated or equivalently fused in the feature dimension to form multimodal fused features. Step 2, Construction of basic logical predicates: Input the multimodal fusion features into a learnable predicate generator to generate multiple sets of basic logical predicates, including relational predicates, deception mechanism predicates, and image-text pairing predicates; Step 3: Derived logic predicate construction: According to the preset logical combination relationship, the basic logical predicates are combined using differentiable fuzzy logic operators to generate derived logic predicates; wherein, some derived logic predicates are obtained by directly combining multiple basic logical predicates, and some derived logic predicates are obtained by further combining basic logical predicates with the generated derived logic predicates; Step 4, Predicate Tokenization and Logical Grouping: Map the scalar values ​​of basic logical predicates and derived logical predicates to vector tokens, and divide them into relation groups, mechanism groups, pairing groups and derived groups according to the semantic function of the predicates; Step 5: Shared Local Bidirectional Mamba In-Group Inference: Input the predicate token sequences from the relation group, mechanism group, pairing group, and derivation group into the parameter-shared local bidirectional Mamba; for the predicate token sequence of each logic group, perform forward selective state space scan and reverse selective state space scan respectively, and fuse the outputs of the two scan directions to obtain a locally enhanced predicate sequence with the same length as the input sequence of the logic group. Step 6: Global Bidirectional Mamba Cross-Group Reasoning: The local enhanced predicate sequences corresponding to the relation group, mechanism group, pairing group, and derivation group are concatenated according to the preset logical group order to form a global logical sequence; the global bidirectional Mamba is used to perform forward and reverse selective state space scanning on the global logical sequence respectively, and the scanning results of the two directions are fused to obtain a structured global reasoning state; the structured global reasoning state is feature-aggregated to obtain a global logical representation; Step 7, Model Training and Fake News Detection: Input the global logical representation into the main classification head to obtain the prediction results of real news and fake news; when there are corresponding fine-grained mechanism labels, set a fine-grained auxiliary classification head during the training phase and optimize the model parameters through joint loss; during the detection phase, use the output of the main classification head as the final real and fake news judgment result.

2. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 1, the specific method is as follows: Input the news text x into the pre-trained text encoder E t The news image I is input into the pre-trained image encoder E. i Global pooling or equivalent feature convergence is performed on the encoding results respectively to obtain the text features F. t and image features F i : Where, d t and d i d represents the text feature dimension and image feature dimension, respectively; the text encoder uses a pre-trained language model capable of outputting contextual semantic representations, and the image encoder uses a pre-trained visual network capable of outputting visual semantic representations; the text encoder uses MacBERT, and the image encoder uses SwinTransformer; PoolText and PoolImage represent the global pooling or equivalent feature convergence operators of the text encoder output and the image encoder output, respectively; the equivalent feature convergence refers to the differentiable convergence operation that converts the sequence features output by the text encoder or the spatial features output by the image encoder into a fixed-dimensional feature vector; F The dimension of the multimodal feature F obtained by concatenating text features and image features is represented by x; x represents the input news text, I represents the input news image corresponding to the news text, and E represents the multimodal feature F. t and E i F represents the pre-trained text encoder and the pre-trained image encoder, respectively. t and F i These represent the text feature vector and image feature vector obtained by the corresponding convergence operator, respectively; Text features F t With image features F i By concatenating the features along the feature dimension, we obtain the unified multimodal feature F: The multimodal features F are input into a shared predicate feature mapping network to form a predicate generation representation that can be used by multiple predicate output heads: Among them, h p This represents the predicate generation representation shared by multiple basic predicate output heads; φ p This represents a predicate generator consisting of one or more fully connected layers, normalization layers, nonlinear activation layers, or combinations thereof; the predicate generation representation simultaneously includes textual semantic information, visual semantic information, and the correspondence between the two in the joint feature space; d p This represents the dimension of the shared predicate generation representation.

3. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 2, the predicate generator forms a predicate generation representation h based on multimodal fusion features, which can be shared by multiple basic predicate output heads. p The dimension of the predicate generation representation is d. p The predicate generator connects to three basic predicate output heads, obtaining the basic predicate probabilities for relational, deception mechanism, and image-text pairing classes, respectively. Among them, W R W M W P These represent the weight parameters of the predicate output heads for relational, deception mechanism, and image-text pairing classes, respectively. R b M b P For the corresponding bias parameter, p R p M p P These are the corresponding predicate probability vectors; relational predicates include four predicates: trusted, weak, contradiction, and uncertain; deception mechanism predicates include six predicates: true, satire, false_connection, imposter, manipulated, and misleading; and image-text pairing predicates include two predicates: pair_match and pair_mismatch, thus forming twelve basic logical predicates; Each predicate output head shares the predicate generation representation obtained from the same multimodal fusion feature, while the output layer maintains independent parameters; the relational predicate output head, the deception mechanism predicate output head, and the image-text pairing predicate output head output 4-dimensional, 6-dimensional, and 2-dimensional predicate probability vectors p, respectively. R p M and p P Accordingly, W R W M and W P The matrix sizes are 4×d p 6×d p and 2×d p b R b M and b P The dimensions are 4, 6, and 2 respectively; softmax represents the normalization exponential function, used to convert the unnormalized scores of each output head into corresponding predicate probability vectors; the relational predicate probability vector p R The components are p in order. trusted p weak p contradiction and p uncertain Deception mechanism class predicate probability vector p M The components are p in order. true p satire p false_connection p imposter p manipulated and p misleading Image-text pairing class predicate probability vector p P The components are p in order. pair_match and p pair_mismatch All of the above probability components are located in [0,1], and the sum of all components of the same predicate probability vector is 1.

4. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 3, differentiable fuzzy logic is used to combine the basic predicates and the derived logic predicates constructed from the basic predicates step by step; fuzzy AND and fuzzy OR are defined as follows: Among them, a u ∈[0,1] represents the truth value of the u-th predicate participating in the combination, and n represents the number of predicates participating in the current fuzzy logic combination; A f and O f Let f represent the fuzzy AND operator and the fuzzy OR operator, respectively, and let f denote the identifier subscript of the fuzzy logic operator. Based on the above operators, eight derived logic predicates are constructed: relation conflict, arbitrary spurious mechanism, truth support, semantic deception, cross-modal deception, contradictory spurious, paired truth support, and paired conflict spurious. These eight derived logic predicates are constructed according to the following differentiable fuzzy logic relations: Where, p RC p FA p TS p SD p CMD p CF p PTS and p PCF These represent relational conflict, arbitrary false mechanism, true support, semantic deception, cross-modal deception, contradictory false, paired true support, and paired conflict false, respectively; the underlying predicate truth value p participating in the above derivational combinations trusted p weak p contradiction p uncertain p true p satire p false_connection p imposter p manipulated p misleading p pair_match p pair_mismatch These represent the truth values ​​of the twelve basic logical predicates, all of which are located in the range [0,1]. FA and p TS In subsequent combinatorial expressions, the derived logic predicates that have been constructed in advance are represented and can continue to be used as inputs to differentiable fuzzy logic operators in successive combinations.

5. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 4, the truth value p of the j-th logical predicate is... j Mapped to d by learnable value projection tok Dimensional token space; will p j Its inverse truth value 1-p j Both are used as input to the value projection, and predicate type embeddings and logical group embeddings are superimposed: Where, φ v Represents a predicate-valued projection network; e j type Let e ​​represent the semantic identity embedding of the j-th predicate. g(j) group This represents the group embedding of the logical group to which the j-th predicate belongs; both the predicate semantic identity embedding and the logical group embedding are trainable parameters with dimension d. tok These are used to identify the specific semantic identity of the predicate and its corresponding logical group; the initial values ​​of the predicate semantic identity embedding and the logical group embedding can be randomly initialized or equivalently trainable initialized, and are updated together with the parameters of the predicate value projection network during model training. The twelve basic predicates and eight derived predicates are organized into the following four groups: where j represents the predicate index, g(j) represents the logical group to which the j-th predicate belongs, and d tok This indicates a unified predicate token that hides a dimension; e j G represents the vector token corresponding to the j-th predicate; R G M G P and G D These represent predicate token sequences for relation groups, mechanism groups, pairing groups, and derivation groups, respectively; p j ∈[0,1];φ v By p j With 1-p j The constructed two-dimensional numerical vector serves as the input, and d is the output. tok dimensional vector; Among them, e trusted e weak e contradiction e uncertain e true e satire e false_connection e imposter e manipulated e misleading e pair_match and e pair_mismatch Each represents a vector token corresponding to one of the twelve basic logical predicates; e RC e FA e TS e SD e CMD e CF e PTS and e PCF G represents the vector tokens corresponding to the eight derived logical predicates: relational conflict, arbitrary spurious mechanism, true support, semantic deception, cross-modal deception, contradictory spurious, paired true support, and paired conflict spurious; R G M G P and G D Let represent the predicate token sequences of the relation group, mechanism group, pairing group, and derivation group, respectively. The number of tokens in each group is 4, 6, 2, and 8, respectively, and the hidden dimension of each predicate token is d. tok .

6. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 5, for any logical group g, intra-group inference is performed using local bidirectional Mamba with shared parameters among the four logical groups: where M L denotes a local Mamba with shared parameters among the four logical groups, Rev(·) denotes reversing the sequence order; G g denotes the input token sequence of the g-th logical group, H g → and H g ← respectively denote the forward scanning result and the reverse scanning result after restoring the original sequence order; the forward output and the reverse output with restored original order are fused, subjected to residual connection and normalization to obtain a local enhanced predicate sequence C g while keeping the number of tokens in this logical group unchanged; W L and b L respectively denote the weight matrix and bias vector for bidirectional state fusion, LN denotes layer normalization, L g denotes the number of tokens in the g-th logical group; W L and its transpose matrix W L T have dimensions of d tok ×2d tok and 2d tok ×d tok respectively; the bidirectional concatenated state is right-multiplied by W in the formula L T to map back to d per token tok dimensions; b L is a d tok -dimensional bias vector, which is broadcast and added by token position; G g , H g → and H g ← all have dimensions of L g ×d tok ; the semicolon ";" in the formula denotes concatenating the states of the two directions along the feature dimension; g∈{R,M,P,D}, where R, M, P, D respectively denote the relation group, mechanism group, pairing group and derivation group, and the corresponding local enhanced predicate sequences C g are respectively denoted as C R , C M , C P and C D ; Among them, C g Let C represent the predicate token sequence of the g-th logical group after shared local bidirectional Mamba enhancement. The locally enhanced predicate sequences corresponding to the relation group, mechanism group, pairing group, and derived group are denoted as C, respectively. R C M C P and C D L g L represents the number of tokens in the g-th logical group. R L M L P and L D They are 4, 6, 2, and 8 respectively; d tok This represents the hidden dimension of each predicate token; therefore, shared local bidirectional Mamba only enhances the context representation within the group, without changing the number of tokens in the corresponding logical group or the hidden dimension of a single token.

7. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 6, the locally enhanced predicate sequences C of the relation group, mechanism group, pairing group, and derivation group are... R C M C P and C D Concatenate them sequentially to form a global logical sequence: Perform forward and reverse scans on the global logical sequence using a global bidirectional Mamba independent of the local inferencer: where C G denotes the global logical sequence obtained by concatenating four sets of locally enhanced predicate sequences, L G denotes its token length, M G denotes the global Mamba, Rev(·) denotes reversing the sequence order, U → and U ← respectively denote the global forward scanning result and the reverse scanning result after restoring the original order, W G and b G respectively denote the weight matrix and bias vector for global bidirectional state fusion, LN denotes layer normalization; H G denotes the cross-group reasoning state that maintains the global sequence structure, h G denotes the compact global logical representation obtained by feature aggregation from H G ; PoolLogic denotes an operator that performs learnable pooling, attention pooling, mean pooling or equivalent feature aggregation on the structured global reasoning state H G , all of the aggregation manners are differentiable operations, and the output dimension thereof is consistent with the hidden dimension of a single predicate token, so as to ensure that the gradient in the training stage can be back-propagated from the global logical representation to the preceding reasoning module; wherein the equivalent feature aggregation refers to a sequence aggregation operation whose input is a sequence state, output is a single global representation with a fixed dimension, and which remains differentiable; W G and its transpose matrix W G T have dimensions of d tok ×2d tok and 2d tok ×d tok respectively; the bidirectionally concatenated global state is right-multiplied by W in the formula G T to map back to d token by token tok dimensions; b G is a d tok -dimensional bias vector, which is broadcast and added by token position; H G maintains L G tokens, and the dimension of each token is d tok , the dimension of h G is d tok ; the semicolon ";" in the formula denotes concatenating the global states of the two directions along the feature dimension; Concat denotes concatenating along the token dimension according to the preset order of the relation group, the mechanism group, the pairing group and the derivation group; C G , U → , U ← and H G all have dimensions of L G ×d tok , and L G =L R +L M +L P +L D ; where d tok L represents the unified hidden dimension of a single predicate token; R L M L P and L D These represent the number of tokens in the relation group, mechanism group, pairing group, and derived group, respectively, with values ​​of 4, 6, 2, and 8; L G This represents the total number of tokens in the global logical sequence.

8. The multimodal fake news detection method based on hierarchical logic predicates and bidirectional Mamba as described in claim 1, characterized in that, In step 7, the binary classification master head and the six-class classification auxiliary head in the training phase are respectively represented by global logic h. G For input: Among them, h G This represents the global logical representation, with dimension d. tok ;d tok This indicates a unified predicate token that hides dimensions; bin and l aux W represents the unnormalized classification scores of the binary primary head and the six-class auxiliary head, with dimensions of 2 and 6, respectively; bin and W aux and b bin and b aux These represent the weight matrix and bias vector of the corresponding classification head, respectively; Among them, W bin The size is 2×d tok b bin The dimension of W is 2; aux The size is 6×d tok b aux The dimension is 6; ŷ and q̂ represent the binary prediction probability vector and the six-class prediction probability vector after softmax normalization, respectively; softmax represents the normalization exponential function, all probability components are located in [0,1], and the sum of all components of the same probability vector is 1. The unnormalized scores and probability vectors for binary classification are arranged in the order of "real news, fake news"; the unnormalized scores and probability vectors for six-class classification are arranged in the order of true, satire, false_connection, imposter, manipulated, misleading; the binary classification master head is retained in both the training and detection phases; the six-class classification auxiliary head only participates in optimization during the training phase with corresponding fine-grained supervised labels, and does not participate in the final category discrimination during the detection phase. The final detection result is determined by the output of the binary classification master head.

Images