Multi-modal fake news detection method and system based on multi-domain learning

Abstract

This invention relates to the fields of information security detection and multi-domain learning technology, specifically a multi-modal fake news detection method and system based on multi-domain learning. The invention includes extracting basic modalities, multi-modalities, and fused sentiment features from news text and images; enhancing basic modal features through adaptive fusion of features within the same domain cluster to construct gated input modal features; generating dual routes by combining domain semantics and sentiment dual-gating mechanisms, and obtaining modal route weights through modulation and fusion; relying on an expert network to decompose domain-invariant and specific features based on the route weights, and achieving feature decoupling through orthogonal loss constraints; finally, fusing the two types of features to obtain a single-modal prediction probability, constructing a collaborative query vector by combining global domain features and sentiment features, and outputting the final fake news detection result through cross-attention adaptive weighted aggregation of probabilities. This invention alleviates the problem of scarce tail-domain samples, achieves domain feature decoupling, and improves the accuracy of fake news detection in multiple domains.

Description

Technical Field

[0001] This invention relates to the fields of information security detection and multi-domain learning technology, specifically to a multimodal fake news detection method and system based on multi-domain learning. Background Technology

[0002] With the rapid development of social media, multimodal fake news fused with text and images is spreading rapidly, posing a serious threat to online cognitive security. Multi-domain fake news detection aims to distinguish the authenticity of news in different topic domains, but existing multimodal multi-domain fake news detection technologies still have three major technical shortcomings: First, real news datasets exhibit a significant long-tail distribution, with sufficient samples in the head domain and scarce samples in the tail domain; during multi-domain joint training, the dominant parameters in the head domain are updated, while the tail domain is prone to negative transfer; traditional style blending enhancement directly performs feature mixing across all domains, and there are semantic gaps between different domains. Forced mixing will produce semantic shifts and destroy the inherent feature distribution of the domains; Second, emotional manipulation is a significant means of spreading fake news, but there are domain-specific biases in emotional expression patterns across different domains. For example, fake news in the social domain tends to be highly arousing and emotionally manipulative, while fake news in the health domain often employs a bland and objective narrative, and even genuine news in the entertainment domain can exhibit strong emotional expression. Most existing technologies treat emotion as a static auxiliary feature, failing to dynamically model the domain-dependent relationship between emotional signals and the truthfulness of news, thus introducing emotional bias and reducing the robustness of cross-domain detection.

[0003] Third, existing multi-domain models generally adopt implicit soft parameter sharing mechanisms, which cannot explicitly decouple domain-invariant general news identification patterns from domain-specific micro-style features. The two types of features become entangled, resulting in feature redundancy and limiting the model's cross-domain generalization ability. Summary of the Invention

[0004] This invention provides a multimodal fake news detection method and system based on multi-domain learning.

[0005] The technical solution of this invention is as follows: A multimodal fake news detection method based on multi-domain learning includes the following steps: S1. Receive news samples to be detected. The news samples to be detected contain text information, image information, and news domain tags. Extract features to obtain basic modality features, multimodal features, and fused sentiment features. The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features. S2. Gated input modal features are processed by attention aggregation to obtain aggregated modal features. They are combined with learnable domain embedding vectors for domain-semantic gating to output basic routes. The fused sentiment features are projected and then processed by sentiment gating to obtain sentiment routes. The sentiment routes are modulated in direction and amplitude. The residuals of the modulated sentiment routes and the basic routes are fused and normalized to obtain the routing weights of the gated input modal features. S3. After the gated input modal features are processed by the expert network and aggregated using routing weights, domain-invariant features and domain-specific features are obtained. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant features and the domain-specific features mutually orthogonal, and the domain-invariant features and domain-specific features are decoupled to obtain the domain-invariant features and domain-specific features of the gated input modal features. S4. By concatenating the domain-invariant and domain-specific features of the gated input modality features, complete features for each modality are obtained. After feature extraction and probability mapping, the prediction probabilities of each single modality are obtained. The domain-specific features of all modalities are aggregated to obtain global domain features. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, adaptive weights for each modality are obtained. The prediction probabilities of each modality are then aggregated by weighting to output the final fake news prediction probability.

[0006] Furthermore, in S1, the style distribution of the basic modal features is adaptively mixed only within the same domain cluster. The process is as follows: For the input text features and image features, the mean and standard deviation of the feature channel dimension style statistics are calculated; reference samples are randomly selected from the same domain cluster, and the style statistics of the anchor sample and the reference sample are convexly combined based on the distribution sampling mixing coefficient. The mixed style statistics are injected into the destylated original features to generate the enhanced basic modal features and expand the tail domain feature distribution.

[0007] Furthermore, in S2, the emotion routing is modulated in both direction and amplitude, as follows: By combining learnable domain-adaptive modulation coefficients with activation functions that have a bidirectional modulation mechanism, the magnitude of emotion is scaled and the modulation amplitude is strictly constrained within a range.

[0008] Furthermore, in S3, the expert network is used to extract domain-invariant features and domain-specific features through domain-invariant expert networks and domain-specific expert networks, respectively.

[0009] Furthermore, in S3, a domain discriminator with a gradient inversion layer is used for adversarial training on domain-invariant features, while a domain discriminator without a gradient inversion layer is used for training on domain-specific features.

[0010] Furthermore, the adaptive weights for each mode in S4 are as follows: Global domain features are concatenated with fused sentiment features, and a collaborative query vector is generated through a pre-defined mapping network. The collaborative query vector is used as the query feature of the cross-attention mechanism, and the complete features of each modality are used as key features and value features. Cross-attention calculation is performed based on the query features, key features, and value features, and the normalized importance weights corresponding to each modality are output.

[0011] Furthermore, the aggregation modal features in S2 are processed as follows: For the feature sequence of gated input modal features, an attention mechanism is used to calculate the importance score of each sequence label in the feature sequence using a pre-defined mapping network. Based on the importance score, the sequence labels in the feature sequence are weighted and aggregated to obtain aggregated modal features.

[0012] Furthermore, the basic modal features in S1 include text features and image features, and the fused sentiment features are generated by fusing text sentiment features and image sentiment features.

[0013] A multimodal fake news detection system based on multi-domain learning includes the following: The multimodal feature extraction and domain enhancement module receives news samples to be detected, which contain text information, image information, and news domain tags. It extracts features to obtain basic modal features, multimodal features, and fused sentiment features. The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features. The dual-route weight generation module obtains aggregated modal features by focusing the gated input modal features, combines them with learnable domain embedding vectors for domain-semantic gating, and outputs basic routes. The fused sentiment features are projected and then processed by sentiment gating to obtain sentiment routes. The sentiment routes are modulated in direction and amplitude, and the residuals of the modulated sentiment routes and the basic routes are fused and normalized to obtain the route weights of the gated input modal features. The expert domain feature decoupling module processes the gated input modal features through an expert network and then aggregates them using routing weights to obtain domain-invariant and domain-specific features. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant and domain-specific features mutually orthogonal, thus decoupling the gated input modal features to obtain the domain-invariant and domain-specific features. The cross-attention collaborative prediction module concatenates the domain-invariant and domain-specific features of the gated input modality to obtain the complete features of each modality. After feature extraction and probability mapping, the prediction probability of each single modality is obtained. The global domain features are obtained by aggregating the domain-specific features of all modalities. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, the adaptive weights of each modality are obtained. The prediction probabilities of each modality are then weighted and aggregated to output the final fake news prediction probability.

[0014] Furthermore, the multimodal feature extraction and domain enhancement module includes a domain style enhancement module, the contents of which are as follows: For the input text features and image features, calculate the mean and standard deviation style statistics of the feature channel dimensions; randomly select reference samples from the same domain cluster, and based on the distribution sampling mixing coefficient, perform a convex combination of the style statistics of the anchor sample and the reference sample, inject the mixed style statistics into the destylated original features, generate the enhanced basic modal features, and expand the tail domain feature distribution.

[0015] The beneficial effects of this invention are as follows: 1. This invention first extracts basic modal, multimodal, and fused sentiment features from news text and images. It then enhances the basic modal features through adaptive fusion of features within the same domain cluster, constructing gated input modal features. A dual-gating mechanism combining domain semantics and sentiment is used to generate dual routes, which are then modally fused to obtain modal route weights. An expert network is used to separate domain-invariant and specific features based on the route weights, achieving feature decoupling through orthogonal loss constraints. Finally, the two types of features are fused to obtain a single-modal prediction probability. A collaborative query vector is constructed by combining global domain features and sentiment features. The probability is then aggregated through cross-attention adaptive weighting to output the final fake news detection result. This invention alleviates the problem of scarce tail-domain samples, achieves domain feature decoupling, and improves the accuracy of fake news detection across multiple domains.

[0016] 2. This invention divides the news domain into several domain clusters and adaptively mixes the style distribution of basic modal features only within the same domain cluster to generate enhanced basic modal features, thereby mitigating the negative transfer caused by the long-tail distribution of multiple domains.

[0017] To address the semantic shift inherent in existing full-domain style mixing methods due to semantic gaps, this invention combines large-scale model semantic priors with manual review to divide semantically compatible domain clusters. It performs intra-cluster mixing and inter-cluster isolation, performing convex combination of style statistics only on basic modal features within each cluster. This expands the tail sample and feature space, avoids cross-cluster contamination, ensures semantic integrity, promotes the transfer of head knowledge to the tail, and improves the performance of identifying fake news at the tail.

[0018] 3. This invention obtains a basic route by aggregating the attention of gated input modal features and combining it with learnable domain embedding for domain-semantic gating. The projected sentiment features are then normalized and fused with the residual of the basic route after sentiment gating and direction / amplitude modulation to generate the routing weights for the gated input modal features. This solves the problem of cross-domain sentiment heterogeneity and eliminates static discrimination bias. The sentiment signal is used as a gating modulation factor, and its amplification, suppression, or inverse cancellation is dynamically controlled by domain-adaptive learnable coefficients. The role of sentiment cues shifts with the domain, no longer simply equating high-arousal sentiment with fake news, thus achieving domain-adaptive utilization of sentiment cues and improving fake news identification performance.

[0019] 4. This invention obtains domain-invariant and domain-specific features by processing the gated input modal features through an expert network and weighting and aggregating them according to routing weights. Orthogonal loss is used to constrain the orthogonality of the two subspaces, thereby achieving feature decoupling. A gradient reversal layer is used to learn the domain-invariant general discrimination mode, while the domain discriminator learns the domain-specific micro-style. Orthogonal loss is used to constrain the orthogonality of the two subspaces, stripping away redundant mutual information and explicitly decoupling to obtain a purer feature representation.

[0020] 5. This invention predicts the probability of a single modality by concatenating domain-invariant and domain-specific features of each modality, and then weights and aggregates the global domain features and fused sentiment features using adaptive weights generated through cross-attention to output the final fake news prediction probability, thereby further improving the overall detection accuracy. The discrimination value of different modalities changes dynamically in different domain scenarios. This invention combines domain information and fused sentiment features to generate a collaborative query vector, and dynamically calculates the adaptive weights of text, image, and multimodal branches through a cross-attention mechanism, weighting and aggregating the independent prediction probabilities of each modality to fully explore the discrimination advantages of each text and image modality. Attached Figure Description

[0021] Figure 1 A radar chart comparing the F1 scores of the three hybrid strategies across nine fine-grained domains of the dataset; Figure 2 The diagram illustrates the domain classification loss and the domain classification results; where a represents the loss based on domain-specific features and the loss based on domain-invariant features, and b represents the domain classification results for each branch. Detailed Implementation

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should also be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0024] This embodiment provides a multimodal fake news detection method based on multi-domain learning, including the following steps: S1. Receive a news sample to be detected, which contains text information, image information, and news domain tags, and extract features to obtain basic modal features, multimodal features, and fused sentiment features; The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features.

[0025] In this embodiment of the invention, the fake news detection task aims to output a prediction result of the authenticity of news. The formal definition of this task is as follows: Assume Indicates inclusion A dataset of news samples. The samples in the dataset are divided by domain, and the domain set is denoted as . , contains Each is an independent field. Given a single news sample. ,in , and These represent the text information, image information, and corresponding domain label information of the sample, respectively. Each sample Associate it with an authenticity tag. ,in Representing real news, This represents fake news.

[0026] Therefore, the goal of multimodal, multi-domain fake news detection is: based on the textual information of news samples. Image information and corresponding domain tag information Learn a classification mapping function This allows for accurate judgment of the authenticity of news.

[0027] Basic modal features, multimodal features, and fused sentiment features are extracted from text and images. The basic modal features include text features and image features, while the fused sentiment features are generated by fusing text sentiment features and image sentiment features. Specifically: 1) Text features: Pre-trained language models have demonstrated powerful context modeling and representation capabilities in natural language processing tasks. To accurately capture deep semantic information in news texts, this invention employs the BERT pre-trained language model based on a Transformer bidirectional encoder structure as the text feature extraction network. For a given news text, text features are extracted. .

[0028] 2) Image features: For visual modality processing, the aim is to extract image features with high generalization ability and deep structural information. The Visual Feature Extraction (MAE) network is used as the visual feature extraction network. Given news image information, image features are extracted. .

[0029] 3) Multimodal features: Although BERT and MAE can extract high-quality single-modal deep semantics respectively, direct fusion often faces the problem of heterogeneity across modal feature spaces because they are pre-trained in independent modal spaces.

[0030] To address this, the CLIP model is introduced to extract naturally aligned multimodal features. By performing self-supervised contrastive learning on large-scale image-text pairs, CLIP successfully maps textual and visual information into the same semantically coexisting joint embedding space, thereby accurately reflecting the closeness of images and text in global semantics.

[0031] Specifically, given news text and news image information, pre-trained CLIP text encoders and image encoders are used to extract synchronized and aligned text and visual features, respectively. and .

[0032] Since these two types of features have been mapped to an isomorphic semantic space during the pre-training phase, their vector distance can directly measure cross-modal relevance. To deeply capture the intrinsic correlation between the two modalities during the fusion process and mitigate the negative impacts introduced by modal ambiguity or noise, we first calculate their cosine similarity in the shared space, using this as a measure of cross-modal semantic consistency. .

[0033] in, and It extracts synchronized aligned text and visual features using pre-trained CLIP text encoders and image encoders, respectively. To score the similarity, This indicates transpose.

[0034] Subsequently, the aligned text and image features are concatenated, and deep cross-modal interaction is performed using a multilayer perceptron (MLP), utilizing similarity scores. Adaptive weighting is applied to the fused features. The final generated multimodal features It can be formalized as: ;in, This indicates a vector concatenation operation.

[0035] 4) Textual sentiment characteristics: This invention extracts multidimensional sentiment features from news texts, given by... A sequence of news texts consisting of words ,in, This indicates the first word in the sequence of news texts. This indicates the first [item] in the news text sequence. Words ( ), This indicates the first [item] in the news text sequence. Each word refers to the last word; features are extracted from five dimensions: sentiment category, sentiment dictionary, sentiment intensity, sentiment score, and auxiliary sentiment information, and then integrated into a unified text sentiment feature.

[0036] For the sentiment category, a publicly available high-precision sentiment classifier is used to calculate... The probability distribution across various emotion categories is used to obtain the emotion category features. .

[0037] For sentiment lexicons, the sentiment expression of a text is usually anchored to specific core words, performing word-level to text-level sentiment aggregation. Assuming the sentiment lexicon contains... Discrete emotions are denoted as emotion sets. ,in This represents the first discrete emotion in the emotion set; This represents the second type of discrete emotion in the emotion set; Represents the first in the set of emotions The last type of discrete emotion.

[0038] For a given Calculate sentiment words In the text Frequency of occurrence in And simultaneously examine its context window. The modifier function within, especially the modifying value of negative words. Modification value of degree adverbs .text In specific emotions The aggregate score can be simply described as: .

[0039] in, Representing text In specific emotions Aggregated scores on; Represents a collection of emotions Any specific emotion in; Represents a given sequence of news texts; Representing text Words in the text; Words expressing emotions exist Frequency of occurrence in; Indicator The context window; Indicator In its context window The value of a word that is modified by a negative word; ) indicates words In its context window The modification value of an adverb of degree.

[0040] Then In all The emotional scores are concatenated to obtain the final emotional lexicon features. .

[0041] Regarding emotional intensity, different words exhibit varying degrees of emotional arousal. This is based on the word intensity weights provided by the emotional lexicon. The word-level scores are weighted and summed to capture the perceived text-level sentiment intensity score. : .

[0042] in, express In specific emotions The text-level sentiment intensity score; This represents the total number of words in the text sequence; For word indexing, ; Represents the first in the text sequence One word; This refers to words provided in the emotion dictionary. The weight of emotional intensity; Indicator In specific emotions Word-level scores; Represents a discrete set of emotions; The calculation of text-level sentiment intensity scores is applicable to sentiment sets. Every emotion in it.

[0043] By concatenating the intensity scores of all emotions, an emotion intensity feature is generated. .

[0044] For sentiment scores, the HowNet sentiment dictionary is used to calculate the macro-sentiment polarity value of the entire news text, i.e., positive, negative, or neutral, which is then used as the polarity density feature. .

[0045] To capture the potential emotional information contained in non-lexical elements, auxiliary sentiment features were constructed by quantifying the frequency of use of online emoticons and punctuation marks, as well as the probability of occurrence of high-frequency emotional interjections in the text. .

[0046] Finally, the five sentiment features mentioned above are concatenated and projected through a fully connected layer with a ReLU activation function to obtain the final text sentiment features. .

[0047] 5) Image emotional features: Compared to text, images can create a strong sense of immersion and realism by directly stimulating the visual center, providing more intuitive and powerful emotional information. This study extracts multidimensional emotional features from news images based on three levels: facial expressions, global color distribution, and local texture disorder.

[0048] For facial emotion features, facial expressions in news photos are a core source of directly conveying emotions. This invention uses the OpenFace 2.0 toolkit to extract potential facial expression features from news images, including facial action units and their high-order geometric representations. Then, a feedforward network (FNN) is used to map and obtain facial emotion features. .

[0049] Regarding color features, the global hue and color scheme of an image often map to specific psychological emotional tones; for example, cool colors are often associated with depression, while highly saturated warm colors often evoke excitement. This invention utilizes a globally accumulated histogram to describe the color distribution characteristics of an image. Let... Quantization level of a specific color in an image Pixel count, Total number of pixels in the image. Frequency of occurrence of each color level. Defined as:

[0050] in, Indicates a specific color quantization level Frequency of occurrence; Indicates the first Each color quantization level; Indicates the position of the first element in the image. Each color quantization level Pixel count; This represents the total number of pixels in the image; The index representing the color quantization level; Indicates the first Each color quantization level; Indicates the position of the first element in the image. Each color quantization level Pixel count.

[0051] To take into account the gradual change in color distribution, Indicates a color level less than or equal to The cumulative sums are used to construct a global cumulative histogram as a color feature. :

[0052] in, Indicates a color level less than or equal to The frequency summation; This indicates the summation condition, i.e., only the index of the ordinal number is considered. Less than or equal to Summing the color quantization levels; Indicates the first The frequency of occurrence of each color level.

[0053]

[0054] in, This represents the constructed global cumulative histogram, i.e., the th... The final color feature vector of a news image; Indicates the first The cumulative sum corresponding to each color quantization level; Indicates the first The cumulative sum corresponding to each color quantization level; Indicates the first This refers to the cumulative sum corresponding to the last color quantization level; This represents the total number of color quantization levels into which the image colors are divided.

[0055] For texture features, the arrangement and high-frequency distribution characteristics of basic textures in an image can effectively reflect the degree of visual clutter and the intensity of emotional signals. This invention uses the Local Binary Pattern (LBP) algorithm to extract spatial texture. For the center pixel in the image... and its neighboring areas Each pixel, through the sign function Perform thresholding judgment: , .

[0056] in, Represents the center pixel Local binary mode encoded value; and These represent the x and y coordinates of the center pixel in the image space, respectively. This represents the total number of pixels in the neighborhood of the center pixel; Indicates the index of the neighboring pixels ( ); Indicates assigning the first The weights of each neighboring pixel; Represents a symbolic function; Indicates the first The pixel value of each neighboring pixel; The pixel value representing the center pixel; Indicates the first The thresholded output result after subtracting the value of the center pixel from the value of each neighboring pixel.

[0057] in, The center pixel value, The values ​​represent neighboring pixels. A statistical histogram of LBP-encoded data is generated across the entire image to produce high-order visual texture features. Finally, the three types of image sentiment features—facial, color, and texture—are concatenated and then input into a fully connected layer to map them into a space isomorphic to the text sentiment, thus obtaining the final image sentiment features. .

[0058] 6) Integrating emotional characteristics: In acquiring text sentiment features With image emotional features Next, a unified fused sentiment feature is constructed. Considering that the sentiment expression of fake news in cyberspace is often complex, in order to preserve the original sentiment semantic distribution of both, the text sentiment features and image sentiment features under isomorphic dimensions are concatenated. Then, a layer of MLP is used for dimensionality reduction, finally generating the fused sentiment feature. .

[0059] The news domain is divided into several domain clusters. Within each domain cluster, the style distribution of the basic modal features is adaptively mixed to generate enhanced basic modal features.

[0060] After acquiring basic features, it is necessary to further address the challenges posed by the long-tail data distribution in multi-domain scenarios. While traditional data augmentation or feature-level style blending methods can alleviate the sample sparsity problem in the tail domain to some extent, there are often significant semantic gaps between different news domains. Blindly blending across these semantic gaps can disrupt the inherent feature distribution of a specific domain, causing semantic shifts. For example, forcing semantically rigorous financial domain features to interact with casual entertainment domain features often generates noisy features that are neither consistent with financial logic nor with the entertainment context.

[0061] To overcome the above limitations, this invention divides the news domain into several domain clusters and adaptively mixes the style distribution of basic modal features only within the same domain cluster to generate enhanced basic modal features, namely the cluster-aware semantic mixing enhancement mechanism CS-Mix.

[0062] This invention, based on the semantic relevance and style compatibility between domains, combines the semantic prior knowledge of large-scale language models with human review to predefined... Each domain is divided into multiple semantically compatible standard domain clusters. Thus, CS-Mix upgrades the traditional random mixing paradigm to a semantically aware directed mixing paradigm, strictly adhering to the principle of intra-cluster mixing and inter-cluster isolation, achieving tail-domain feature enhancement while preserving cross-domain semantic consistency.

[0063] Specifically, define the domain cluster mapping. ,in For the entire domain set, Represents the division A semantically compatible domain cluster. Among them, This represents a set containing all semantically compatible domain clusters; This represents the total number of domain clusters that have been divided. Indicates the division of the first A semantically compatible domain cluster; Indicates the division of the first A semantically compatible domain cluster. Given the text feature representation of the input. ,in, , , Given batch size, number of feature channels, and sequence length, we first calculate the mean and standard deviation of style statistics along the channel dimension: , .

[0064] in, Indicates the feature representation of the input text The mean of style statistics; Indicates the feature representation of the input text The standard deviation of style statistics; The length of the sequence representing text features; This represents the index along the sequence length dimension. ; Indicates the feature representation of the input text In the sequence dimension Feature slices at each location; To prevent division by zero errors, a very small constant term is included.

[0065] For the domain Anchor point sample ,like From the same semantic cluster Randomly sample another reference sample : .

[0066] in, This represents the reference sample obtained from the sampling process; This indicates that random sampling is performed from the set on the right. This represents any candidate text feature sample in the set; This represents a mapping function that retrieves the domain of a given sample. Indicates the number of anchor point samples Each domain cluster.

[0067] Subsequently, in order to perform smooth style transfer within the cluster, a mixing coefficient is sampled from the Beta distribution. , To control the degree of blending hyperparameters, a convex combination of the style statistics of the anchor point and the reference sample is performed to generate the blended target style parameters. and : , .

[0068] in, This represents the target style standard deviation parameter after mixing; This represents the target style mean parameter after mixing; This represents the mixing coefficient obtained by sampling from the Beta distribution; and The standard deviation and mean of the style statistics of the anchor sample text features are represented, respectively. and These represent the standard deviation and mean of the style statistics of the text features of the sampled reference samples, respectively.

[0069] Finally, the blended style statistics are reinjected into the destylated anchor features to obtain the enhanced text features. :

[0070] in, This represents the enhanced text features generated after style blending injection; Indicates the original anchor text features The process of performing destylization.

[0071] For image features sampled within the same neighborhood cluster, enhanced image features are generated using the same feature statistics and convex combination mechanism. Gated input modal features include enhanced basic modal features and multimodal features.

[0072] By strictly limiting the mixing of styles Within the mapping range, CS-Mix enriches the distribution of tail domain data while effectively reducing cross-domain style pollution between unrelated domains, thus improving the overall generalization ability.

[0073] Therefore, the cluster-aware semantic hybrid enhancement mechanism proposed in this invention effectively alleviates the negative transfer problem caused by the long-tail distribution of multi-domain data. Simultaneously, this invention combines large-model semantic priors with manual review to divide semantically compatible domain clusters, strictly implementing intra-cluster mixing and inter-cluster isolation, performing style statistics convex combination mixing only on basic modal features within a cluster. While expanding the tail-domain feature samples and enriching the tail-domain feature space, it avoids feature pollution from cross-cluster irrelevant domains, ensuring the semantic integrity of the original domain. This allows the model to fully utilize head-domain knowledge to migrate to the sample-scarce tail-domain, significantly improving the performance of fake news identification in the tail domain.

[0074] S2. Gated input modal features are processed through attention aggregation to obtain aggregated modal features. These are then combined with learnable domain embedding vectors for domain-semantic gating to output basic routes. Emotional features are fused, projected, and then processed through emotion gating to obtain emotional routes. The emotional routes are modulated in direction and amplitude. The residuals of the modulated emotional routes and the basic routes are fused and normalized to obtain the routing weights of the gated input modal features.

[0075] After mitigating the feature sparsity problem in the tail domain using CS-Mix, it is necessary to further map these robust multimodal features to a specific expert network for deep decoupling. The spread of fake news is often accompanied by specific emotional manipulation strategies, and these emotional styles vary across different domains. For example, high-arousal emotions are often false signals of fake news in the social domain, but may represent genuine emotional resonance in the sports and entertainment domains.

[0076] However, most existing multi-domain models neglect this crucial domain dependency variability. To accurately model this variable cross-domain sentiment dependency, this invention achieves basic routing by aggregating the attention of gated input modal features and combining them with learnable domain embeddings for domain-semantic gating. The projected sentiment features are then normalized and fused with the residuals of the basic routing after sentiment gating and direction / amplitude modulation to generate the routing weights for the gated input modal features—the Emotion-Aware Dual-Gated Modulation Mechanism (EDGM). As the routing core of the entire decoupling framework, EDGM uses sentiment information as a priori routing information and dynamically calculates and modulates the activation weights of subsequent expert networks through joint modulation via domain-semantic gating and bidirectional sentiment gating pathways.

[0077] Specifically, the first step is to construct a domain-semantic gating system to capture content and domain dependencies. Taking the text modality as an example, to capture the most discriminative core semantic content, an attention mechanism is introduced, utilizing an MLP network. The importance score of each sequence token is calculated, and then weighted and aggregated to obtain semantic features. : .

[0078] in, This represents the semantic features obtained through weighted aggregation. Indicates the total number of sequence markers; Indicates the sequence number index of the sequence marker. Indicates enhanced text features The first in One sequence marker; This represents the multilayer perceptron network used to calculate the importance scores of each sequence label.

[0079] Furthermore, in order to model the domain-specific context in multi-domain learning, this invention introduces a domain embedding vector for each independent domain. The model will be used for the dataset. Each domain learns its own domain-specific representation. Then, to achieve deep interaction between domain knowledge and semantic features, domain embeddings are implemented. With semantic features The data are concatenated and input into a domain-semantic gating network. The routes generated by this gated network, i.e., the basic routes Logits, : .

[0080] in, It is an FNN. Its parameters. In this way, when making basic routing decisions, the gating network can not only keenly capture the objective content of the news, but also take into account its domain context.

[0081] Secondly, a two-way emotion modulation gating system is constructed to capture the domain specificity of emotional information. Firstly, emotional features are... The data is fed into a nonlinear projection layer called Projector, and then input into an emotion-gated network. In the process, the sentiment routing Logits are calculated. : .

[0082] in, and These are the learnable weight matrix and bias term of the projection layer, respectively. This represents the SiLU activation function. For FNN The parameters.

[0083] Given that the correlation between sentiment information and the veracity of fake news depends on the specific domain, this invention utilizes a learnable domain-adaptive modulation coefficient matrix. , contains Domain-specific scalars By combining this coefficient with a bidirectional modulation mechanism By combining activation functions, it is possible to scale the Logits of sentiment routing by magnitude and strictly constrain the adjustment range. Within this range, this effectively prevents sentiment features from excessively dominating semantic features in noisy domains, ensuring the robustness of the system. Finally, the scaled sentiment routing Logits are compared with... Perform Logit-level residual fusion and utilize The function is normalized to generate a routing weight vector to guide subsequent hybrid expert network aggregation. That is, text feature gating weights: .

[0084] The aforementioned fusion mechanism utilizes domain-specific learnable coefficients. and The synergistic operation of activation functions endows the model with directional adaptability, enabling it to dynamically learn and quantify the sentiment relevance of a specific domain, determining whether to positively utilize or negatively suppress sentiment information. when This indicates that sentiment information in this field is positively correlated with fake news detection, and sentiment features will be amplified and actively utilized in the same direction.

[0085] when This indicates that the emotional fluctuations in this field are mainly irrelevant noise, thus adaptively shielding this part of the interfering information.

[0086] when When the data shows that sentiment information in this field is negatively correlated with fake news detection, the model will suppress this potential sentiment bias in the opposite direction.

[0087] The dual-gated routing in this step ensures both the fundamental judgment capability based on pure multimodal semantics and the transformation of sentiment features into an adaptive residual modulation signal, enabling precise dynamic correction of expert-assigned weights. The output text feature gating weights... The decoupling process will be directly passed to domain experts to guide the adaptive aggregation of the expert group. Using the same approach, the corresponding gating weights can be obtained from the image and multimodal branches. and Furthermore, the routing weights of gated input modal features include , , ,in, This represents the image branch gating weights calculated based on the image modal features; This represents the multimodal branch gating weights calculated for the multimodal fusion features.

[0088] This invention addresses the challenge of cross-domain emotional heterogeneity and eliminates static emotional discrimination bias by aggregating attentional features of gated input modalities and combining them with learnable domain embeddings for domain-semantic gating. The projected emotional features are then normalized and fused with the residuals of the basic routes after emotional gating and direction / amplitude modulation, generating routing weights for the gated input modalities. Fake news commonly employs emotional manipulation, but the effectiveness of emotional signals in distinguishing truth from falsehood shifts with domain; for example, fake news in the health domain often uses a bland, objective narrative. This invention extracts textual and image emotional features separately, fusing them to obtain fused emotional features. Using the emotional signal as a gating modulation factor, and leveraging domain-adaptive learnable coefficients, it dynamically controls the amplification, suppression, or inverse cancellation of the emotional signal, moving beyond simply equating high-arousal emotions with fake news and achieving domain-adaptive utilization of emotional cues.

[0089] S3. After the gated input modal features are processed by the expert network and aggregated using routing weights, domain-invariant features and domain-specific features are obtained. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant features and the domain-specific features mutually orthogonal, thus decoupling to obtain the domain-invariant features and domain-specific features of the gated input modal features.

[0090] After receiving the adaptive routing weights output by S2, these weights are used to guide the feature aggregation of subsequent expert networks. However, most existing multi-domain networks rely on implicit soft parameter sharing mechanisms. These mechanisms cannot effectively decouple the general news patterns that do not change with the domain from the news styles specific to each domain in high-dimensional feature spaces, which can easily lead to different expert networks learning overlapping and redundant mutual information.

[0091] To overcome this bottleneck of representation ambiguity, this invention proposes a domain expert representation decoupling process. This is achieved by introducing explicit adversarial constraints and orthogonal losses to decouple domain-invariant features from domain-specific features.

[0092] Single-expert networks often focus on a specific domain, and the extracted features typically contain only localized information, failing to comprehensively cover the complex features of multi-domain news content. For comprehensiveness, this invention employs a hybrid expert architecture (MoE). Domain-invariant expert networks and domain-specific expert networks extract domain-invariant and domain-specific features from the data, respectively, to model multi-domain relationships. To address the heterogeneity of features across different modalities, TextCNN is used as the expert for the text modality, CNN as the expert for the image modality, and MLP as the expert for the multimodal modality. The network's underlying layers deploy... Unchanging experts in various fields and Experts in their respective fields, among whom and This is a hyperparameter.

[0093] Taking text modality as an example, input They will be equally distributed to both the domain-invariant expert network and the domain-specific expert network. Subsequently, the routing weights are determined based on the gated input modal features output by S2. Multiple features extracted by a hybrid expert network are dynamically weighted and aggregated to initially extract domain-invariant features. Domain-specific characteristics .

[0094] Furthermore, decoupling can be achieved through different discriminator paths. and The specific process is as follows: Domain-invariant feature: To enable domain-invariant branches to learn cross-domain general knowledge, the concept of adversarial networks is introduced. It is fed into a neighborhood discriminator with a gradient inversion layer (GRL). In the middle. This discriminator acts as The classifier outputs a... A dimensional vector. During forward propagation, GRL behaves as an identity mapping; while during backward propagation, it multiplies the gradient by a negative constant. This reverses the gradient direction. Its adversarial classification loss is defined as: .

[0095] in, Represents the adversarial classification loss for the domain-invariant branch; This indicates the calculation of mathematical expectation; Indicates the distribution of the training data; Representation of domain-invariant features and their corresponding real-world domain tags The data pairs are composed of data from the training data distribution. Obtained from sampling in the middle; This represents the total number of categories in the independent domain; The index representing the domain category. ; Represents the true domain label of the sample; This indicates the indicator function, when predicting the category index. Equal to real domain labels When, the function takes the value Otherwise, the value is ; This represents the gradient inversion layer (GRL) operation. This represents the domain discriminator; in this embodiment of the invention, the adversarial classification loss is implemented using the standard cross-entropy loss in PyTorch, with the base of the loss function being e.

[0096] In this process, the discriminator strives to identify the domain to which the current news belongs, while the feature extractor updates its parameters in reverse using GRL, attempting to "deceive" the discriminator. This explicit adversarial training forces the domain-invariant expert network to learn deeper, more general features for distinguishing between true and false news. These features capture common fake news patterns across all domains that do not change with the domain, stripping away domain-specific styles and biases. Essentially, it removes domain specificity, leaving only domain indifference. This pure domain-invariant feature is the foundation for achieving strong generalization capabilities, enabling robust detection patterns to be smoothly transferred from data-rich head domains to tail domains, effectively mitigating the negative transfer problem under long-tail distributions.

[0097] Specifically, domain-specific features: Meanwhile, the aggregated domain-specific features... It will be directly sent to the domain discriminator In this branch, there are no GRL constraints. The classification loss function is... Defined as: In this embodiment of the invention, the base of the loss function is e.

[0098] The network is forced to extract features that allow the discriminator to classify it accurately. This is because general knowledge has already been... Sharing, at this time Instead of needing to carry a universal feature space, it can focus entirely on learning the subtle fake news clues unique to the learning field.

[0099] Finally, in order to eliminate redundant mutual information between the two features and maximize the representation capacity, further... and An orthogonal loss constraint was applied. Furthermore, the loss function uses the norm to normalize the cross-correlation of the two features, forcing the two feature subspaces to be mutually orthogonal: .

[0100] This step effectively decouples domain-invariant and domain-specific features. Using the same method, the corresponding domain-invariant and domain-specific features can be obtained from image features and multimodal features, respectively. , , and Furthermore, the domain-invariant and domain-specific features of gated input modal characteristics include... , , , , and ,in, This represents the neighborhood-invariant features decoupled from image features; This represents the domain-specific features decoupled from image features; This represents the domain-invariant features decoupled from multimodal features; This represents the domain-specific features decoupled from multimodal features.

[0101] This invention addresses the feature entanglement and redundancy issues inherent in traditional implicit soft parameter sharing by employing adversarial training and joint orthogonal loss. Traditional multi-domain models rely on soft parameter sharing, resulting in severe feature redundancy due to the entanglement between domain-invariant general discrimination patterns and domain-specific micro-styles, which restricts the model's generalization ability. However, this invention, based on a hybrid expert architecture, utilizes a gradient inversion layer to construct adversarial constraints for learning domain-invariant features and extracting cross-domain general true / false news discrimination patterns; it also utilizes a general domain discriminator to learn domain-specific micro-style features; and finally, it constrains the orthogonality of the two types of feature subspaces through orthogonal loss, stripping away redundant mutual information and explicitly decoupling the two types of features. Compared to implicit sharing methods, this yields a cleaner feature representation.

[0102] S4. By concatenating the domain-invariant and domain-specific features of the gated input modality features, complete features for each modality are obtained. After feature extraction and probability mapping, the prediction probabilities of each single modality are obtained. The domain-specific features of all modalities are aggregated to obtain global domain features. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, adaptive weights for each modality are obtained. The prediction probabilities of each modality are then aggregated by weighting to output the final fake news prediction probability.

[0103] In the multimodal fake news detection of this invention, the reliability and identification value of different modalities often exhibit a dynamic imbalance. Domain attributes characterize the objective narrative background of the news, while sentiment features reveal its subjective dissemination intention. Together, these two factors determine the preferred mediums for fake information.

[0104] Therefore, in order to more comprehensively quantify and address modal uncertainty in multi-domain scenarios, a sentiment-domain collaborative decision-making process is defined. This process involves concatenating domain-invariant and domain-specific features of each modality to predict the probability of a single modality. Then, the global domain features and fused sentiment features are weighted and aggregated using adaptive weights generated across attention to output the final probability of fake news prediction.

[0105] By concatenating the domain-invariant and domain-specific features of the gated input modal features, complete features for each modality are obtained. After feature extraction and probability mapping, the predicted probabilities for each single modality are obtained. Specifically: First, complete features for each modality are constructed using domain-invariant and domain-specific features from the three branches: text, image, and multimodal. , , Secondly, the complete features of these three dimensions are fed into the classifiers of each modality to obtain the probability of predicting fake news from an independent perspective. , and : .

[0106] Subsequently, average pooling is performed on the domain-specific features of the three modalities. Aggregation yields global domain features :

[0107] Next, the global domain features With emotional characteristics The vectors are concatenated and mapped to a joint collaborative query vector using an MLP. The model treats it as a query matrix across attention mechanisms. At the same time, a matrix composed of complete features from three modalities will be generated. as a bond matrix Sum matrix Both are input into the cross-attention calculation, and the corresponding normalized importance weights are output. , , : .

[0108] in, and It is a learnable linear projection matrix. This is a scaling factor to prevent gradient vanishing.

[0109] Finally, these adaptive weights are weighted and aggregated with the predicted probabilities output by each modality-independent classifier to output the final fake news prediction probability. : .

[0110] in, This represents the final probability of the model's prediction of fake news. , and These represent the normalized importance weights of the text modality, image modality, and multimodal branches, respectively, calculated through cross-attention computation. and These represent the probability of predicting fake news from independent perspectives of the text branch, image branch, and multimodal branch, respectively.

[0111] In this embodiment of the invention, in order to enhance the ability of each independent perspective to detect fake news, the text perspective is calculated separately. Visual perspective and multimodal perspective The binary cross-entropy (BCE) loss between the predicted probability and the true label is denoted as... Simultaneously, the final predicted probability and the true label output during the sentiment-domain collaborative decision-making process are also calculated. The main classification BCE loss between .

[0112] Furthermore, to achieve effective decoupling between domain-invariant and domain-specific features in each modal space, a corresponding domain decoupling loss is defined. For each modal branch... Adversarial classification loss with domain-invariant features was applied to each. Domain-specific classification loss And the orthogonal constraint loss aimed at eliminating redundant mutual information between the two. .

[0113] Combining all the above optimization objectives, the overall loss function of the method of this invention, i.e., the EMD-FND model, is... The formal definition is as follows: .

[0114] in, These are the weights for multi-branch assisted detection tasks. It is the weight for global domain resistance loss. These are the weights of the global domain classification loss. This is the penalty weight for the global orthogonality constraint loss. During training, the model minimizes this penalty weight using the backpropagation algorithm. It updates all network parameters collaboratively in an end-to-end manner.

[0115] This invention predicts the probability of a single modality by concatenating domain-invariant and domain-specific features of each modality, and then weights and aggregates the global domain features and fused sentiment features using adaptive weights generated through cross-attention to output the final fake news prediction probability, thereby further improving the overall detection accuracy. The discrimination value of different modalities changes dynamically in different domain scenarios. This invention combines domain information and fused sentiment features to generate a collaborative query vector, and dynamically calculates the adaptive weights of text, image, and multimodal branches through a cross-attention mechanism, weighting and aggregating the independent prediction probabilities of each modality to fully explore the discrimination advantages of each text and image modality.

[0116] This embodiment provides a multimodal fake news detection system based on multi-domain learning, including the following: The multimodal feature extraction and domain enhancement module receives news samples to be detected, which contain text information, image information, and news domain tags. It extracts features to obtain basic modal features, multimodal features, and fused sentiment features. The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features. The dual-route weight generation module obtains aggregated modal features by focusing the gated input modal features, combines them with learnable domain embedding vectors for domain-semantic gating, and outputs basic routes. The fused sentiment features are projected and then processed by sentiment gating to obtain sentiment routes. The sentiment routes are modulated in direction and amplitude, and the residuals of the modulated sentiment routes and the basic routes are fused and normalized to obtain the route weights of the gated input modal features. The expert domain feature decoupling module processes the gated input modal features through an expert network and then aggregates them using routing weights to obtain domain-invariant and domain-specific features. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant and domain-specific features mutually orthogonal, thus decoupling the gated input modal features to obtain the domain-invariant and domain-specific features. The cross-attention collaborative prediction module concatenates the domain-invariant and domain-specific features of the gated input modality to obtain the complete features of each modality. After feature extraction and probability mapping, the prediction probability of each single modality is obtained. The global domain features are obtained by aggregating the domain-specific features of all modalities. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, the adaptive weights of each modality are obtained. The prediction probabilities of each modality are then weighted and aggregated to output the final fake news prediction probability.

[0117] In the experiments of this invention, based on the Weibo dataset, the cluster-aware semantic mixing enhancement mechanism CS-Mix of this invention was compared with two variants in a fine-grained performance manner: one is the baseline model with the module removed w / o CS-Mix, and the other is the strategy Global-Mix, which removes intra-cluster constraints and performs global blind mixing directly across all domains.

[0118] Reference Figure 1 The paper presents radar charts comparing the F1 scores of three hybrid strategies across nine fine-grained domains in the dataset. Specifically, the negative transfer phenomenon under the long-tail distribution is significant in the baseline model without any feature mixing. After introducing the Global-Mix strategy, although the model's performance recovers in some tail domains, its overall performance is not stable. Optimal cross-domain generalization is achieved based on domain cluster partitioning. This fully demonstrates that by strictly restricting style synthesis to synthesized samples within semantically compatible domain clusters, feature contamination between irrelevant domains is avoided while enriching the tail domains, thus achieving a joint performance improvement across multiple domains.

[0119] The analysis of decoupling effect based on adversarial constraints specifically involves using a domain adversarial learning mechanism to eliminate differences between different domains in the domain-invariant branch, while retaining specific information about domain-specific characteristics in the domain-specific branch, thereby enhancing the model's generalization ability.

[0120] To further explore the specific effects of explicit adversarial constraints on decoupling, the domain classification loss and performance of the EMD-FND method of this invention during dataset training were recorded and plotted. For detailed results, please refer to... Figure 2 'a' shows the loss based on domain-specific features. And loss based on domain-invariant features. The diagram shows the domain classification results for each branch; b.

[0121] Judging from the trend of the loss curve, the classification loss of GRL without gradient inversion layer is... The decline indicates that the specific branches are increasingly accurately grasping the domain-specific features; the adversarial classification loss uses a gradient inversion layer (GRL). The sustained increase in domain-invariant features after the initial adversarial game suggests that these features are becoming increasingly difficult to distinguish between different domains, implying a narrowing of domain-specific differences. The classification F1 score of the domain-specific branch consistently rises during training and always outperforms the results of the domain-invariant branch, which remains at a lower level. In summary, the method of this invention effectively extracts both domain-invariant and domain-specific features, achieving high-quality decoupling.

Claims

1. A multi-modal fake news detection method based on multi-domain learning, characterized in that, Includes the following steps: S1. Receive news samples to be detected. The news samples to be detected contain text information, image information, and news domain tags. Extract features to obtain basic modality features, multimodal features, and fused sentiment features. The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features. S2. Gated input modal features are processed by attention aggregation to obtain aggregated modal features. They are combined with learnable domain embedding vectors for domain-semantic gating to output basic routes. The fused sentiment features are projected and then processed by sentiment gating to obtain sentiment routes. The sentiment routes are modulated in direction and amplitude. The residuals of the modulated sentiment routes and the basic routes are fused and normalized to obtain the routing weights of the gated input modal features. S3. After the gated input modal features are processed by the expert network and aggregated using routing weights, domain-invariant features and domain-specific features are obtained. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant features and the domain-specific features mutually orthogonal, and the domain-invariant features and domain-specific features are decoupled to obtain the domain-invariant features and domain-specific features of the gated input modal features. S4. By concatenating the domain-invariant and domain-specific features of the gated input modality features, complete features for each modality are obtained. After feature extraction and probability mapping, the prediction probabilities of each single modality are obtained. The domain-specific features of all modalities are aggregated to obtain global domain features. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, adaptive weights for each modality are obtained. The prediction probabilities of each modality are then aggregated by weighting to output the final fake news prediction probability.

2. The multi-domain learning based multi-modal fake news detection method according to claim 1, wherein, In S1, the style distribution of basic modal features is adaptively mixed only within the same domain cluster. The process is as follows: For the input text features and image features, the mean and standard deviation of the feature channel dimension style statistics are calculated; reference samples are randomly selected from the same domain cluster, and the mixing coefficients are sampled based on the distribution. The style statistics of the anchor sample and the reference sample are convexly combined, and the mixed style statistics are injected into the destylated original features to generate the enhanced basic modal features and expand the tail domain feature distribution. 3.The multi-modal fake news detection method based on multi-domain learning according to claim 1, wherein, In S2, the emotion routing is modulated in both direction and amplitude, as follows: By combining learnable domain-adaptive modulation coefficients with activation functions that have a bidirectional modulation mechanism, the magnitude of emotion is scaled and the modulation amplitude is strictly constrained within a range.

4. The multimodal fake news detection method based on multi-domain learning according to claim 1, characterized in that, In S3, the expert network extracts domain-invariant features and domain-specific features through domain-invariant expert networks and domain-specific expert networks, respectively.

5. The multimodal fake news detection method based on multi-domain learning according to claim 4, characterized in that, In S3, a domain discriminator with a gradient inversion layer is used for adversarial training on domain-invariant features, while a domain discriminator without a gradient inversion layer is used for training on domain-specific features.

6. The multimodal fake news detection method based on multi-domain learning according to claim 1, characterized in that, The adaptive weights for each mode in S4 are as follows: Global domain features are concatenated with fused sentiment features, and a collaborative query vector is generated through a pre-defined mapping network. The collaborative query vector is used as the query feature of the cross-attention mechanism, and the complete features of each modality are used as key features and value features. Cross-attention calculation is performed based on the query features, key features, and value features, and the normalized importance weights corresponding to each modality are output.

7. The multimodal fake news detection method based on multi-domain learning according to claim 1, characterized in that, The aggregation modal features in S2 are processed as follows: For the feature sequence of gated input modal features, an attention mechanism is used to calculate the importance score of each sequence label in the feature sequence using a pre-defined mapping network. Based on the importance score, the sequence labels in the feature sequence are weighted and aggregated to obtain aggregated modal features.

8. The multimodal fake news detection method based on multi-domain learning according to claim 1, characterized in that, The basic modal features in S1 include text features and image features, while the fused sentiment features are generated by fusing text sentiment features and image sentiment features.

9. A multi-domain learning-based multimodal fake news detection system, used to implement the multi-domain learning-based multimodal fake news detection method as described in claim 1, characterized in that, Includes the following: The multimodal feature extraction and domain enhancement module receives news samples to be detected, which contain text information, image information, and news domain tags. It extracts features to obtain basic modal features, multimodal features, and fused sentiment features. The news domain is divided into several domain clusters. The style distribution of basic modal features is adaptively mixed only within the same domain cluster to generate enhanced basic modal features. The gated input modal features include enhanced basic modal features and multimodal features. The dual-route weight generation module obtains aggregated modal features by focusing the gated input modal features, combines them with learnable domain embedding vectors for domain-semantic gating, and outputs basic routes. The fused sentiment features are projected and then processed by sentiment gating to obtain sentiment routes. The sentiment routes are modulated in direction and amplitude, and the residuals of the modulated sentiment routes and the basic routes are fused and normalized to obtain the route weights of the gated input modal features. The expert domain feature decoupling module processes the gated input modal features through an expert network and then aggregates them using routing weights to obtain domain-invariant and domain-specific features. Orthogonal loss constraints are applied to make the subspaces of the domain-invariant and domain-specific features mutually orthogonal, thus decoupling the gated input modal features to obtain the domain-invariant and domain-specific features. The cross-attention collaborative prediction module concatenates the domain-invariant and domain-specific features of the gated input modality to obtain the complete features of each modality. After feature extraction and probability mapping, the prediction probability of each single modality is obtained. The global domain features are obtained by aggregating the domain-specific features of all modalities. The global domain features are concatenated with the fused sentiment features to generate a collaborative query vector. After cross-attention processing, the adaptive weights of each modality are obtained. The prediction probabilities of each modality are then weighted and aggregated to output the final fake news prediction probability.

10. The multimodal fake news detection system based on multi-domain learning according to claim 9, characterized in that, The multimodal feature extraction and domain enhancement module includes a domain style enhancement module, which includes the following: For the input text features and image features, calculate the mean and standard deviation of the feature channel dimension style statistics; randomly select reference samples from the same domain cluster, and based on the distribution sampling mixing coefficient, perform a convex combination of the style statistics of the anchor sample and the reference sample, inject the mixed style statistics into the destylated original features, generate the enhanced basic modal features, and expand the tail domain feature distribution.

Images