False news detection method based on progressive field adaptive multi-view learning

Abstract

The invention provides a false news detection method based on progressive field adaptive multi-view learning. The method comprises the following steps: step 1, constructing a multi-modal field adaptive data set; step 2, single-mode feature extraction based on pre-training knowledge enhancement; step 3, constructing a visual and language cross-modal comparison visual angle; step 4, domain-aware dynamic routing and feature fusion; 5, carrying out progressive course learning optimization training; and step 6, inputting the final news representation subjected to field enhancement into a full-connection classifier, calculating the authenticity probability of the news sample, and outputting a final dichotomy prediction result through a Sigmoid activation function. According to the method, general knowledge of a pre-training model is reserved through efficient fine tuning of parameters, heterogeneity characteristics of a specific field are effectively captured, adaptive fusion of multi-modal information in different fields is achieved through a dynamic routing mechanism, and robustness and accuracy of false news detection in a cross-field scene are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, and in particular relates to a method for detecting fake news based on progressive domain-adaptive multi-view learning. Background Technology

[0002] With the widespread application of mobile internet and social media platforms, the ecosystem of digital information dissemination and consumption habits have undergone significant changes, with multimodal data (text, images, videos, etc.) gradually becoming the main carrier of information dissemination. While this multimedia format greatly improves the efficiency of information acquisition, its decentralized publishing mechanism also facilitates the growth and spread of misinformation. Multimodal fake news detection, as a key research direction, aims to construct efficient and reliable detection models by jointly analyzing textual semantics and visual content to identify fake content in cyberspace. By accurately capturing cross-modal semantic conflicts and forgery traces, this technology has significant application value in maintaining the authenticity of online information, ensuring public safety, and building a healthy online ecosystem.

[0003] Given the broad application prospects of multimodal fake news detection, researchers have proposed a large number of detection models to try to solve the core challenges in the field, including fusion methods based on traditional deep learning, detection methods based on pre-trained basic models, and multi-domain adaptive methods. However, existing multimodal fake news detection technologies still have many shortcomings: (1) Existing multimodal alignment mechanisms are often limited to simple feature splicing or shallow interaction, making it difficult to capture the hidden deep semantic conflicts between images and text (such as "image and text do not match" or satirical exaggeration), resulting in limited reasoning ability when facing complex deep forgery or fine-grained semantic tampering. (2) Real-world news data exhibits significant domain heterogeneity, with huge differences in forgery methods and data distribution in different domains (such as politics and entertainment). Existing models have poor generalization performance in cross-domain scenarios and are difficult to adapt to the detection needs of unseen domains. (3) End-to-end fine-tuning paradigms based on large language models or visual-language models are prone to "shortcut learning," that is, the model tends to rely on biased vocabulary or image style artifacts in specific domains for judgment, rather than truly understanding the factual logic across modalities. (4) Most existing multi-domain processing methods adopt static or globally shared modal fusion strategies, ignoring that the degree of dependence of different domains on graphic information is dynamically changing. They cannot adaptively adjust the decision weights of each modality according to the characteristics of a specific domain, thus limiting the detection accuracy of the model in complex scenarios. Summary of the Invention

[0004] Purpose of the Invention: The technical problem to be solved by this invention is to address the following shortcomings in existing multimodal fake news detection technologies: (1) Existing methods are difficult to capture hidden deep semantic conflicts between cross-modal content (such as "image and text mismatch"), and their reasoning ability is limited when faced with complex deepfake techniques; (2) Real-world news data has significant domain heterogeneity, and existing models mostly adopt static fusion strategies, which cannot adaptively adjust modal dependencies according to the characteristics of specific domains, resulting in poor cross-domain generalization performance; (3) End-to-end fine-tuning paradigms are prone to causing models to fall into shortcut learning, relying on surface statistical bias rather than factual logic. To this end, this invention provides a fake news detection method (ProDFND) based on progressive domain adaptive multi-view learning, including the following steps:

[0005] Step 1, Construct a multimodal domain-adaptive dataset: Collect fake news detection data containing text and image content, clean the data and divide it into different domain categories, including politics, entertainment and social life, and divide the dataset into training set, validation set and test set; Step 2, Single-modal feature extraction based on pre-trained knowledge enhancement: Use a large language model with frozen parameters as the text backbone network and a visual and language pre-trained model with frozen parameters as the visual backbone network, respectively. In the Transformer layer of the backbone network, a trainable domain-specific low-rank adapter LoRA is embedded in parallel. Domain-specific knowledge is injected into the general features through the low-rank adapter LoRA, thereby extracting domain-aware text sequence features and domain-aware visual sequence features respectively;

[0006] Step 3, Constructing a visual and linguistic cross-modal contrastive perspective: A lightweight query transformer, Q-Former, is introduced to extract visual sequence representations aligned with the text space. A bidirectional cross-attention alignment network is constructed, guided by textual and visual features respectively, to reconstruct each other's feature representations. The difference vector between the original and reconstructed features is then calculated to generate cross-modal contrastive features capable of capturing deep semantic conflicts. Step 4, Domain-Aware Dynamic Routing and Feature Fusion: A domain-aware dynamic routing network is constructed. A learnable domain embedding vector is initialized for each domain. Global features from the text, visual, and cross-modal contrastive perspectives are concatenated with the domain embedding vector. A gating mechanism adaptively calculates the decision weights of each perspective within the current domain. Based on these decision weights, multi-perspective features are weighted and aggregated to obtain the domain-enhanced final news representation.

[0007] Step 5, Progressive Course Learning Optimization Training: Construct a three-stage progressive training strategy that includes single-modal warm-up, cross-modal alignment, and global fusion optimization. By dynamically adjusting the weight coefficients of the loss function with the number of training steps, the model parameters are guided to gradually optimize from local feature perception to global logical decision-making.

[0008] Step 6, Fake News Prediction Output: Input the domain-enhanced final news representation into the fully connected classifier, calculate the authenticity probability of the news sample, and output the final binary classification prediction result through the Sigmoid activation function.

[0009] Step 2 includes: given the input news text The pre-trained language model Qwen2.5-7B is used as the basic text encoder. For a specific field Text sequence features perceived by the computational domain :

[0010] ,

[0011] ,

[0012] in, For general text semantic representation, LoRA is a domain-specific low-rank adapter for the text side. and Given two low-rank matrices, Represents the space of real numbers, rank Much smaller than the text feature dimension , Include Feature vectors of a text token;

[0013] Given a news image as input The visual language pre-trained model BLIP-2 is used as the backbone of the visual encoder. Visual sequence features perceived in the computational domain :

[0014] ,

[0015] ,

[0016] in, For general visual semantic representation, For the vision side, a domain-specific low-rank adapter (LoRA) is used. The LoRA adds domain-specific biases to the frozen visual encoder output. Include The feature vector of a visual patch.

[0017] Step 3 includes:

[0018] Step 3.1: Initialize a set of learnable query embeddings using Q-Former, a lightweight query transformer in the visual language pre-trained model BLIP-2. Through cross-attention mechanism and frozen visual features Interacting with the text allows visual features to be mapped to a text alignment space, resulting in a visual sequence representation. :

[0019] ,

[0020] Step 3.2, utilize the learnable linear projection layer of text feature mapping. Learnable linear projection layer for visual feature mapping Text features Visual sequence representation Mapped to a uniform alignment dimension ;

[0021] Step 3.3: Construct a bidirectional cross-attention mechanism to calculate the visual features reconstructed guided by the text context. and text features reconstructed guided by visual content. :

[0022] ,

[0023] ,

[0024] in, represents the dimension of the attention head in the multi-head attention mechanism, Softmax is the normalization exponential function, and M represents the matrix transpose operation;

[0025] Step 3.4: Calculate the absolute difference vector between the original features and the reconstructed features, and concatenate the difference vector with the aligned features. Then, aggregate the concatenation using a multilayer perceptron (MLP) to generate the final cross-modal contrastive representation. :

[0026] ,

[0027] ,

[0028] in, This represents a vector concatenation operation. This represents the absolute difference vector, obtained by taking the absolute value of each element. Used to explicitly represent the inconsistency between text and image semantics.

[0029] In step 3.3, the number of heads in the bidirectional cross-attention mechanism is set to 8 to balance the adaptive aggregation of multi-view features with computational efficiency.

[0030] Step 4 includes:

[0031] Step 4.1, analyze text sequence features and visual sequence features Global average pooling is performed along the sequence dimension to obtain a global text vector of fixed dimensions. and global visual vectors For cross-modal contrast representation, directly let the global contrast vector... ;

[0032] Step 4.2: Determine the domain to which the current input sample belongs. Matching the corresponding domain embedding vector By concatenating the global vectors from the three perspectives with the domain embedding vectors, a complete context vector is constructed. :

[0033] ;

[0034] Step 4.3: Input the context vector into the dynamic routing network consisting of two multilayer perceptrons and the GELU activation function. The normalized gate weights are output through the normalized exponential function Softmax. :

[0035] ,

[0036] in, These are the learnable parameters for the routing network. The contributions from textual, visual, and contrastive perspectives are respectively calculated, and the following conditions are met: , Represents the GELU activation function;

[0037] Step 4.4: Perform a weighted summation of the global vector based on the gating weights to obtain the domain-enhanced news representation. :

[0038] .

[0039] In step 4.3, the dimension of the hidden layer in the dynamic routing network is set to 192;

[0040] In step 5, the loss function is... Defined as with training steps Dynamically changing weighted sum form:

[0041] ,

[0042] in, They represent the prediction probabilities based on the text perspective. Visual perspective prediction probability Cross-modal contrastive perspective prediction probability and the probability of prediction from the final fusion perspective Calculate the binary cross-entropy loss; These are the time-varying coefficients that control the learning weights of single-mode, contrastive-mode, and fused-mode, respectively.

[0043] In step 5, the three-stage progressive training strategy includes: the first stage is a single-modal warm-up stage, corresponding to a training step number of... In the first phase, the dynamic routing network parameters are frozen, and only the domain-specific low-rank adapter (LoRA) parameters are optimized, with the weighting coefficient set to [value missing]. The second stage is the cross-modal alignment stage, corresponding to the number of training steps. In the second stage, the cross-modal alignment task is activated, with weight coefficients... A linear growth strategy is employed to guide the model in capturing features where the text and images do not match.

[0044] ,

[0045] The third stage is the global fusion optimization stage, corresponding to the number of training steps. In the third stage, the routing network and classifier were fully fine-tuned, and task weights were integrated. Significant improvement; the weights of single-mode and contrastive modes decay linearly to 0.2.

[0046] ,

[0047] in, Set to 20% of the total steps. Set to 60% of the total steps. This represents the maximum number of training steps.

[0048] In step 5, the typical value is the rank setting of the domain-specific low-rank adapter LoRA. The scaling factor is set to The random dropout rate is set to 0.1; the maximum length of the text input is truncated to 256; and the resolution of the input image is adjusted to 224×224 pixels.

[0049] The present invention also provides an electronic device for detecting fake news, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the method when executing the instructions.

[0050] The present invention also provides a non-volatile computer-readable storage medium storing computer program instructions, which, when executed by a processor, are used to implement the method described herein.

[0051] Compared with the prior art, the present invention has the following significant advantages:

[0052] (1) The present invention adopts the parameter-efficient fine-tuning paradigm of “freezing the trunk + domain-specific low-rank adapter LoRA”. While preserving the general world knowledge of the large model, it effectively captures the heterogeneous features of specific domains through the low-rank adapter, which significantly alleviates the negative transfer problem in cross-domain generalization.

[0053] (2) The present invention designs a domain-aware dynamic routing network that can adaptively and dynamically adjust the decision weights of text, visual and cross-modal comparison perspectives according to the domain attributes (such as politics or entertainment) of the input sample, thus solving the problem that traditional static fusion strategies are difficult to adapt to complex "domain-modal" interaction relationships.

[0054] (3) This invention introduces an explicit cross-modal contrast perspective and a bidirectional cross-attention mechanism, which can accurately capture fine-grained semantic conflicts between text and images (such as irony and inconsistency between text and images), and breaks through the inference bottleneck caused by simple feature splicing.

[0055] (4) This invention constructs a progressive course learning strategy from easy to difficult. By dynamically adjusting the loss weight in stages, it effectively avoids the model from getting trapped in local optima or shortcut learning too early in complex multimodal tasks, thereby improving the stability and convergence performance of training.

[0056] (5) The method proposed in this invention has been extensively validated on three widely used real-world benchmark datasets: Weibo, Weibo-21, and FineFake. The experimental results show that the proposed method significantly outperforms the current state-of-the-art (SOTA) baseline methods in key metrics such as accuracy, F1 score, and domain generalization ability, especially demonstrating superior performance when dealing with complex samples with high domain heterogeneity and fine-grained image and text tampering. Attached Figure Description

[0057] Figure 1 This is a flowchart of the method of the present invention.

[0058] Figure 2 This is a model framework diagram of the system of the present invention.

[0059] Figure 3 This is a flowchart for the text-guided module.

[0060] Figure 4 This is a flowchart of the cross-modal interaction layer.

[0061] Figure 5 This is a diagram illustrating a progressive course learning optimization strategy. Detailed Implementation

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0063] This implementation provides a fake news detection method based on progressive domain-adaptive multi-view learning. The overall architecture of the system model provided in this embodiment is as follows: Figure 1 As shown, the model mainly includes three core components: (1) a pre-trained knowledge-enhanced domain-aware multi-view analysis module; (2) a domain-aware multi-view fusion module; and (3) a progressive training strategy module.

[0064] like Figure 1 As shown, the method includes the following steps:

[0065] Step 1, Data Collection: Collect training, evaluation, and testing datasets for multimodal fake news detection. This dataset includes data from both text and image modalities, along with corresponding domain labels. The dataset is cleaned and segmented to meet the needs of subsequent feature extraction. "Multimodal" refers to both text and visual modalities.

[0066] Step 2, Multimodal Feature Extraction: For the input news data, features are extracted using pre-trained base models. A pre-trained language model with frozen parameters, Qwen2.5-7B, is used as the base text encoder to extract general text semantics, and a pre-trained visual language model with frozen parameters, BLIP-2, is used as the base visual encoder to extract general visual semantics. To capture domain-specific heterogeneous features, a trainable domain-specific low-rank adapter (LoRA) is inserted in parallel into the encoder's Transformer layer to extract domain-aware text sequence features. and visual sequence features .

[0067] Step 3, Constructing a Cross-Modal Contrast Perspective: Utilizing the lightweight query transformer Q-Former in the visual language pre-trained model BLIP-2, visual sequence representations aligned with the text space are generated through the interaction of learnable query embeddings and frozen visual features. A bidirectional cross-attention alignment network is constructed to compute the visual features reconstructed guided by the text context. and text features reconstructed guided by visual content By calculating the difference vector between the original features and the reconstructed features, and then aggregating them via a multilayer perceptron, cross-modal contrastive features capable of capturing image-text consistency and conflict are generated. .

[0068] Step 4, Domain-Aware Dynamic Route Construction: Utilizing Text Sequence Features Visual sequence features and cross-modal contrast features Global average pooling is performed to obtain a global view vector. A learnable domain embedding vector is initialized for each domain, and concatenated with the global view vectors from each perspective to construct a context vector containing micro-level content information and macro-level domain priors. This context vector is then input into a domain-aware dynamic routing network to adaptively calculate the gating weights for text, visual, and cross-modal contrastive perspectives in the current decision. Based on the previously generated gating weights, the multi-view features are weighted and aggregated to obtain the final domain-enhanced news representation. This step uses a dynamic routing mechanism to adaptively adjust the contribution of each modality based on the domain attributes of the input samples (such as politics or entertainment), thereby effectively capturing domain-specific forgery patterns while maintaining general features.

[0069] Step 5, Progressive Course Learning Optimization: This invention employs a three-stage progressive training strategy to optimize the model. The specific implementation steps for each stage are as follows: Stage 1 (Single-modal warm-up): Freeze the routing network, optimizing only the parameters of the domain-specific low-rank adapter LoRA to establish the model's basic perception capabilities for specific domain text and images. Stage 2 (Cross-modal alignment): Gradually introduce cross-modal contrast tasks, guiding the model to focus on image-text consistency through linearly increasing weights, capturing deep semantic conflicts. Stage 3 (Global Fusion Optimization): Shift the training focus to the domain-aware dynamic routing network, comprehensively fine-tuning the routing network and classifier. By dynamically adjusting the loss function weights, joint optimization from local feature perception to global decision-making is achieved. This results in a domain-enhanced news representation. The data is input into a fully connected classifier, where the probability of news authenticity is calculated using the Sigmoid activation function, generating fake news detection results. Simultaneously, the model is trained under supervised supervision using the binary cross-entropy loss function to ensure its detection accuracy and robustness in complex cross-domain scenarios.

[0070] Step 1 includes:

[0071] In the research and verification of multimodal fake news detection technology, data collection and construction are key steps in realizing model training, performance evaluation, and generalization ability testing. To ensure the scientific validity of the technical solution described in this invention, the applicability of the data, and the effectiveness of the verification model in solving domain heterogeneity problems, this implementation selects three representative, highly differentiated, and widely recognized real-world multimodal benchmark datasets: Weibo, Weibo-21, and FineFake.

[0072] The Weibo dataset, proposed by Wang et al., is one of the classic benchmarks in the field of multimodal fake news detection, and is described in detail in their paper "Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, KishlayJha, Lu Su, and Jing Gao. EANN: Event Adversarial Neural Networks for Multi-Modal Fake News Detection. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 840–849". The data in this dataset was collected from Sina Weibo, China's largest social media platform. Its notable characteristics include a large number of early social media rumors, and the text and images are often highly inflammatory. To evaluate the model's detection performance in different topic scenarios, this invention strictly divides the dataset into nine different domains (including politics, entertainment, business, health, society, conflict, etc.). As shown in Table 1, the Weibo dataset contains 3,226 real news items and 3,654 fake news items, providing data support for verifying the model's basic detection capabilities in standard multi-domain scenarios.

[0073] To further verify the robustness of the model in the face of data distribution shifts and changes in time span, this invention introduces the Weibo-21 dataset, published by Nan et al., and described in detail in their paper "Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. MDFEND: Multi-domain Fake NewsDetection. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (CIKM). 3343–3347". Although this dataset also originates from the Sina Weibo platform, its data collection spans a wider time span, covering more recent social hot topics, and its sample size and domain distribution show a significant distribution shift compared to the Weibo dataset. As shown in Table 1, the Weibo-21 dataset is also organized into 9 corresponding domains, containing a total of 5,544 samples, of which 2,753 are real news and 2,791 are fake news. The purpose of introducing this dataset is to test whether the model can learn domain-invariant forgery patterns independent of specific dataset biases.

[0074] The FineFake dataset was chosen to test the model's ability to perform across language environments and at the level of fine-grained semantic understanding. Proposed by Zhou et al., and introduced in their paper "Zhaoyu Zhou, Xinyu Zhang, LiyangZhang, Jiahang Liu, Shiyao Wang, Zhaocheng Liu, and Philip S. Yu. FineFake: AKknowledge-Enriched Dataset for Fine-Grained Multi-Domain Fake News Detection. In Proceedings of the ACM Web Conference 2024 (WWW). 3432–3443". FineFake is an English multimodal dataset covering six core themes: politics, entertainment, business, health, society, and conflict. Its unique advantage lies in providing fine-grained semantic annotation based on "knowledge elements" and containing a large amount of long-text news and high-resolution images. As shown in Table 1, the FineFake dataset contains a total of 14,373 samples, which is crucial for verifying whether the model can bridge the "semantic gap" and accurately identify subtle inconsistencies between images and text (such as images being incorrectly attributed to unrelated events).

[0075] Table 1

[0076]

[0077] Step 2 includes:

[0078] In processing input multimodal news data, the feature extraction module of this invention aims to utilize the general knowledge of large pre-trained models and combine it with domain-specific information to generate high-quality unimodal representations. The feature extraction module mainly comprises two parts: text-perspective feature extraction and visual-perspective feature extraction, such as... Figure 2 As shown, the specific process is as follows:

[0079] 2.1 Text-based Feature Extraction

[0080] For the input news text T, this embodiment uses the pre-trained language model Qwen2.5-7B as the basic text encoder, denoted as... To capture the specific semantic features of news domains (such as politics, entertainment, and health) while retaining the powerful general language understanding capabilities of the base model, this invention embeds a trainable, domain-specific low-rank adapter (LoRA) in parallel within the Transformer layer of the encoder, denoted as... This module injects domain knowledge into general features through efficient parameter fine-tuning. The text feature extraction process is defined by the following formula:

[0081] ,

[0082] ,

[0083] in Represents the input text sequence; This represents a basic text encoder function whose parameters remain frozen during training, and whose output is a general text semantic representation. ; Indicates targeting a specific field The trainable text-side domain-specific low-rank adapter LoRA, this module consists of two low-rank matrices and The product of these components has dimensions respectively. and ; The rank of the domain-specific low-rank adapter LoRA, and satisfying ; Dimensions of text features; This indicates the final result, which integrates general knowledge and domain-specific knowledge. The text sequence features of specific information have the following dimensions: , The length of the text sequence.

[0084] 2.2 Visual Perspective Feature Extraction

[0085] For the input news image V, this embodiment uses the visual language pre-trained model BLIP-2 as the backbone of the basic visual encoder, denoted as BLIP-2. The encoder includes a visual transformer and a lightweight query transformer, Q-Former. To effectively identify visual forgery artifacts specific to different domains (such as image manipulation in entertainment news or low-resolution artifacts in breaking news), a domain-specific visual LoRA adapter is also introduced. This module superimposes a domain-specific bias onto the frozen visual encoder output. The visual feature extraction process is defined by the following formula:

[0086] ,

[0087] ,

[0088] in This represents the input image data; The underlying visual encoder function represents parameter freezing, and its output is a general visual semantic representation. ; Indicates the target field LoRA, a trainable vision-side domain-specific low-rank adapter; This represents the final domain-aware visual sequence features, with dimensions of [dimensionality]. , The length of the visual patch sequence. This refers to the dimension of visual features. Through the above steps, this invention obtains domain-aware feature representations for both textual and visual modalities. and These two sets of features not only preserve the world knowledge of large pre-trained models, but also incorporate fine-grained forgery patterns of specific domains in a parameter-efficient manner, providing high-quality input for subsequent cross-modal alignment and dynamic fusion.

[0089] Step 3 includes:

[0090] This invention constructs a cross-modal fine-grained comparison perspective, aiming to deeply explore the alignment relationship between images and text in a fine-grained semantic space to capture hidden forgery clues such as "image-text mismatch" and "semantic conflict." Figure 3 As shown, this step is specifically divided into three stages: visual sequence alignment, bidirectional cross-attention inference, and differential feature aggregation. The specific implementation process is as follows:

[0091] 3.1 Visual Sequence Alignment

[0092] To obtain visual representations aligned with the text semantic space, this invention utilizes the lightweight query transformer Q-Former in the BLIP-2 architecture of the visual language pre-trained model. First, a set of learnable query embedding vectors is initialized. Through the cross-attention mechanism and the frozen visual features extracted in step 2 This process involves interacting with the text. It maps visual information to a semantic space consistent with the text, generating visual sequence representations. :

[0093] ,

[0094] in, The query vector is randomly initialized and optimized with the network. This represents the number of queries.

[0095] 3.2 Bidirectional Cross-Attention Reasoning

[0096] To simulate the bidirectional verification logic of humans when verifying facts—"verifying images with text" and "verifying text with images"—this invention constructs a bidirectional cross-attention alignment network. First, it utilizes two learnable projection layers... and Text features Visual sequence representation Mapping to a unified alignment dimension. Then, two parallel cross-attention mechanisms are constructed: one uses text representation as the query vector and visual representation as the key / value pair, computing the visual features reconstructed guided by the text context. Secondly, using visual representations as query vectors and textual representations as key-value pairs, textual features reconstructed based on visual content are calculated. The calculation formula is as follows:

[0097] ,

[0098] ,

[0099] in This represents the dimension of the attention head in a multi-head attention mechanism. This is the scaling factor; It is a normalized exponential function; This represents the matrix transpose operation.

[0100] 3.3 Aggregation of Differential Features

[0101] To explicitly capture semantic conflicts between text and images, this invention calculates the absolute difference vector between the original features and the reconstructed features. The difference vector is then concatenated with the aligned features. Finally, a multilayer perceptron (MLP) is used to nonlinearly aggregate the features, resulting in a cross-modal contrastive representation that includes deep logical consistency information. :

[0102] ,

[0103] ,

[0104] in, This indicates a feature concatenation operation. This represents the element-wise absolute value operation. It can effectively reflect the logical authenticity of the input news, providing a key basis for subsequent decision-making and integration.

[0105] Step 4 includes:

[0106] This invention constructs a domain-aware dynamic routing network, aiming to address the difference in modal dependence among news from different domains (e.g., entertainment news relies heavily on visual impact, while political news relies heavily on textual logic), and to achieve adaptive fusion of multi-perspective features. Figure 4As shown, this step is specifically divided into three stages: global perspective representation compression, dynamic routing weight calculation, and adaptive feature aggregation. The specific implementation process is as follows:

[0107] 4.1 Global Perspective Representation Compression and Domain Embedding

[0108] Text features extracted in the preceding steps and visual features All are variable-length sequence representations. To perform efficient global routing decisions, this invention first employs a global average pooling strategy to compress features along the sequence dimension, resulting in fixed-dimensional global text view vectors. and global visual view vector For cross-modal contrastive features Since its output via the Multilayer Perceptron (MLP) is already in vector form, its global view vector can be directly set. Furthermore, in order to explicitly model the macroscopic characteristics of different domains, this invention provides each domain in the dataset with... Initialize a learnable domain embedding vector, denoted as . This vector will adaptively encode the modality preference pattern of the domain during training.

[0109] 4.2 Dynamic Route Weight Calculation

[0110] To comprehensively consider both the micro-level information of the current sample and the macro-level prior knowledge of its domain, this invention concatenates the global view vectors from the three perspectives mentioned above with the domain embedding vector to construct a complete context vector. :

[0111] ,

[0112] Subsequently, a domain-aware dynamic routing network was utilized. The contribution of each viewpoint is calculated. The network consists of two multilayer perceptron (MLP) layers. The middle layer is equipped with the GELU activation function, and the output layer is normalized using the Softmax normalization function to generate a gated weight vector specific to the current sample. The calculation formula is as follows:

[0113] ,

[0114] in and These are the weight matrices and bias vectors for the first and second layers of the routing network, respectively. Represents the GELU activation function; These represent the weights of textual, visual, and cross-modal contrastive perspectives in the current decision-making process, and satisfy the following conditions: This mechanism ensures that the model can dynamically "route" attention based on the domain to which the sample belongs, quantifying the decision importance of each perspective.

[0115] 4.3 Adaptive Feature Aggregation

[0116] Based on the generated gating weights, this invention adaptively weights and aggregates global features from three perspectives to obtain the final domain-enhanced news representation. :

[0117] ,

[0118] Through the aforementioned dynamic routing mechanism, the model can intelligently adjust the decision weight of each modality according to the domain attributes of the input samples, thereby effectively capturing forgery patterns in specific domains while maintaining general feature representations, and significantly enhancing the model's robustness to domain heterogeneity.

[0119] Step 5 includes:

[0120] To address the gradient instability and "shortcut learning" problems (i.e., the model prematurely relies on high-level fusion features while neglecting the quality of low-level single-modal representations) that are prone to occur in end-to-end training of models containing large-scale pre-trained backbones and multi-layered interaction modules, this invention designs a progressive curriculum learning optimization strategy, such as... Figure 5 As shown. This strategy guides the model to optimize along a cognitive path "from perception to reasoning, and then to decision-making" by dynamically adjusting the weights of the loss function. This step is specifically divided into three parts: loss function construction, phased progressive training, and final prediction output. The specific implementation process is as follows:

[0121] 5.1 Loss Function Construction

[0122] To enhance the discriminative power of each view, this invention applies binary cross-entropy (BCE) loss to the text view, visual view, cross-modal contrast view, and the final fused view. The total loss function... Defined as with training steps The dynamically changing weighted sum is calculated using the following formula:

[0123] ,

[0124] in They represent text feature-based Visual features Cross-modal pair bits and final fusion characteristics Predicted probability and true label The BCE loss calculated between; These are the time-related weighting coefficients for controlling the learning focus of single-modal, contrastive, and fusion modalities, respectively.

[0125] 5.2 Phased and progressive training

[0126] Let the total number of training steps be This invention divides the training process into three progressive stages to achieve optimization from easy to difficult:

[0127] Phase 1: Single-mode preheating ( ),

[0128] At this stage, dynamic routing network parameters are frozen, focusing on optimizing the domain-specific low-rank adapter (LoRA). Weighting coefficients are set to... This setup aims to allow the model to prioritize establishing basic perceptual capabilities for domain-specific text and images without being disturbed by complex cross-modal interactions. It is usually set at 30% of the total number of steps.

[0129] Phase Two: Cross-Modal Alignment ( ),

[0130] After establishing a solid foundation in single-modal computing, cross-modal comparison tasks are gradually introduced. A linear growth strategy is employed to guide the model to focus on text-image consistency, thereby learning to capture deep semantic conflicts. The weight calculation formula is as follows:

[0131] , This phase lasts until (Usually 70% of the total number of steps).

[0132] Phase 3: Global Fusion Optimization ),

[0133] Finally, the training focus was shifted to a domain-aware dynamic routing network. The weights for the fusion task were significantly increased, forcing the model to learn how to schedule features from different perspectives based on domain priors, while the weights for unimodal and contrastive tasks gradually decreased. The weight calculation formula is as follows:

[0134] ,

[0135] at the same time and The value is linearly decayed to 0.2. This strategy ensures that the underlying feature extractor remains stable during the final optimization stage, while maximizing the performance of the global decision.

[0136] 5.3 Final Prediction Output

[0137] After the above progressive training, the domain-enhanced news representation obtained in step 4 will be used. The input is fed into a fully connected classifier, and the probability of the news being true is calculated using the Sigmoid activation function. This is used to determine whether the input news is real or fake news:

[0138] ,

[0139] By comparing this prediction result with the true label, and combining it with the total loss function mentioned above for backpropagation, the model parameters are updated and optimized.

[0140] Baseline Comparison: The following models were selected as baselines for comparison of model performance in this implementation to verify the effectiveness of the present invention: (1) MMoE: adopts a multi-gated hybrid expert network architecture, which models the correlation and differences between different domains by sharing the underlying expert network and using a gating network for specific tasks. (2) MoSE: designed for sequence data, introduces a sequence expert model to specifically handle domain dependencies and temporal features in long texts. (3) MDFEND: automatically aggregates feature representations of multiple domain experts using a domain gating network, which is a representative work based on single-modal multi-domain methods. (4) M³FEND: introduces meta-learning and memory mechanisms on the basis of multi-expert architecture to store and transfer domain-invariant knowledge patterns and improve adaptability to domains with few samples. (5) KATMF: uses a knowledge-enhanced Transformer to extract deep semantics and uses a domain adversarial training mechanism to eliminate domain-specific noise interference, thereby guiding the model to learn cross-domain invariant multimodal discriminative features. (6) MMDFND: explores the dynamic interaction mechanism of multimodal features in multi-domain scenarios, aiming to capture both the consistency between modalities and the differences between domains. (7) DAMMFND: It improves cross-domain detection performance by decoupling common and unique features of the domain through domain adversarial learning and multi-view fusion network.

[0141] As shown in Tables 2, 3 and 4, the method proposed in this invention outperforms the performance of all baseline models on the three public datasets, thus verifying the effectiveness of this invention.

[0142] Table 2

[0143]

[0144] As shown in Tables 2, 3 and 4, the method proposed in this invention outperforms the performance of all baseline models on the three public datasets, thus verifying the effectiveness of this invention.

[0145] Table 3

[0146]

[0147] Table 4

[0148]

[0149] This invention provides a method for detecting fake news based on progressive domain-adaptive multi-view learning. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for detecting fake news based on progressive domain-adaptive multi-view learning, characterized in that, Includes the following steps: Step 1, Construct a multimodal domain-adaptive dataset: Collect fake news detection data containing text and image content, clean the data and divide it into different domain categories, including politics, entertainment and social life, and divide the dataset into training set, validation set and test set; Step 2, Single-modal feature extraction based on pre-trained knowledge enhancement: Use a large language model with frozen parameters as the text backbone network and a visual and language pre-trained model with frozen parameters as the visual backbone network, respectively. In the Transformer layer of the backbone network, a trainable domain-specific low-rank adapter LoRA is embedded in parallel. Domain-specific knowledge is injected into the general features through the low-rank adapter LoRA, thereby extracting domain-aware text sequence features and domain-aware visual sequence features respectively; Step 3, Constructing a visual and linguistic cross-modal contrastive perspective: A lightweight query transformer, Q-Former, is introduced to extract visual sequence representations aligned with the text space. A bidirectional cross-attention alignment network is constructed, guided by textual and visual features respectively, to reconstruct each other's feature representations. The difference vector between the original and reconstructed features is then calculated to generate cross-modal contrastive features capable of capturing deep semantic conflicts. Step 4, Domain-Aware Dynamic Routing and Feature Fusion: A domain-aware dynamic routing network is constructed. A learnable domain embedding vector is initialized for each domain. Global features from the text, visual, and cross-modal contrastive perspectives are concatenated with the domain embedding vector. A gating mechanism adaptively calculates the decision weights of each perspective within the current domain. Based on these decision weights, multi-perspective features are weighted and aggregated to obtain the domain-enhanced final news representation. Step 5, Progressive Course Learning Optimization Training: Construct a three-stage progressive training strategy that includes single-modal warm-up, cross-modal alignment, and global fusion optimization. By dynamically adjusting the weight coefficients of the loss function with the number of training steps, the model parameters are guided to gradually optimize from local feature perception to global logical decision-making. Step 6, Fake News Prediction Output: Input the domain-enhanced final news representation into the fully connected classifier, calculate the authenticity probability of the news sample, and output the final binary classification prediction result through the Sigmoid activation function.

2. The method according to claim 1, characterized in that, Step 2 includes: given the input news text The pre-trained language model Qwen2.5-7B is used as the basic text encoder. For a specific field Text sequence features perceived by the computational domain : ， ， in, For general text semantic representation, LoRA is a domain-specific low-rank adapter for the text side. and Given two low-rank matrices, Represents the space of real numbers, rank Smaller than text feature dimension , Include Feature vectors of a text token; Given a news image as input The visual language pre-trained model BLIP-2 is used as the backbone of the visual encoder. Visual sequence features perceived in the computational domain : ， ， in, For general visual semantic representation, For the vision side, a domain-specific low-rank adapter (LoRA) is used. The LoRA adds domain-specific biases to the frozen visual encoder output. Include The feature vector of a visual patch.

3. The method according to claim 2, characterized in that, Step 3 includes: Step 3.1: Initialize a set of learnable query embeddings using Q-Former, a lightweight query transformer in the visual language pre-trained model BLIP-2. Through cross-attention mechanism and frozen visual features Interacting with the text allows visual features to be mapped to a text alignment space, resulting in a visual sequence representation. : ， Step 3.2, utilize the learnable linear projection layer of text feature mapping. Learnable linear projection layer for visual feature mapping Text features Visual sequence representation Mapped to a uniform alignment dimension ; Step 3.3: Construct a bidirectional cross-attention mechanism to calculate the visual features reconstructed guided by the text context. and text features reconstructed guided by visual content. : ， ， in, represents the dimension of the attention head in the multi-head attention mechanism, Softmax is the normalized exponential function, and M represents the matrix transpose operation; Step 3.4: Calculate the absolute difference vector between the original features and the reconstructed features, and concatenate the difference vector with the aligned features. Then, aggregate the concatenation using a multilayer perceptron (MLP) to generate the final cross-modal contrastive representation. : ， ， in, This represents a vector concatenation operation. This represents the absolute difference vector, obtained by taking the absolute value of each element. Used to explicitly represent the inconsistency between text and image semantics.

4. The method according to claim 3, characterized in that, In step 3.3, the number of heads in the bidirectional cross-attention mechanism is set to 8.

5. The method according to claim 4, characterized in that, Step 4 includes: Step 4.1, analyze text sequence features and visual sequence features Global average pooling is performed along the sequence dimension to obtain a global text vector of fixed dimensions. and global visual vectors For cross-modal contrast representation, directly let the global contrast vector... ; Step 4.2: Determine the domain to which the current input sample belongs. Matching the corresponding domain embedding vector By concatenating the global vectors from the three perspectives with the domain embedding vectors, a complete context vector is constructed. : ； Step 4.3: Input the context vector into the dynamic routing network consisting of two multilayer perceptrons and the GELU activation function. The normalized gate weights are output through the normalized exponential function Softmax. : ， in, These are the learnable parameters for the routing network. The contributions from textual, visual, and contrastive perspectives are respectively calculated, and the following conditions are met: , This represents the GELU activation function; Step 4.4: Perform a weighted summation of the global vector based on the gating weights to obtain the domain-enhanced news representation. : 。 6. The method according to claim 5, characterized in that, In step 5, the loss function is... Defined as with training steps Dynamically changing weighted sum form: ， in, They represent the prediction probabilities based on the text perspective. Visual perspective prediction probability Cross-modal contrastive perspective prediction probability and the probability of prediction from the final fusion perspective Calculate the binary cross-entropy loss; These are the time-varying coefficients that control the learning weights of single-mode, contrastive-mode, and fused-mode, respectively.

7. The method according to claim 6, characterized in that, In step 5, the three-stage progressive training strategy includes: the first stage is a single-modal warm-up stage, corresponding to a training step number of... In the first phase, the dynamic routing network parameters are frozen, and only the domain-specific low-rank adapter LoRA parameters are optimized, with the weighting coefficient set to [value missing]. The second stage is the cross-modal alignment stage, corresponding to the number of training steps. In the second stage, the cross-modal alignment task is activated, with weight coefficients... A linear growth strategy is employed to guide the model in capturing features where the text and images do not match. ， The third stage is the global fusion optimization stage, corresponding to the number of training steps. In the third stage, the routing network and classifier were fine-tuned, with the weights of single-modal and contrastive modalities linearly decaying to 0.

2. ， in, Set to 20% of the total steps. Set to 60% of the total steps. This represents the maximum number of training steps.

8. The method according to claim 7, characterized in that, In step 5, the rank of the domain-specific low-rank adapter LoRA is set to... The scaling factor is set to The random dropout rate is set to 0.1; the maximum length of the text input is truncated to 256; and the resolution of the input image is adjusted to 224×224 pixels.

9. An electronic device for detecting fake news, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to, when executing the instructions, implement the method as described in any one of claims 1 to 8.

10. A non-volatile computer-readable storage medium, characterized in that, The storage medium stores computer program instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 8.

Images