A multi-field fake news detection method based on hierarchical latent field modeling

Abstract

The application discloses a multi-field fake news detection method based on hierarchical potential field modeling, and belongs to the technical field of fake news detection. The method comprises the following steps: obtaining multi-angle multi-modal feature embedding of to-be-detected news by using a pre-training feature extractor; enhancing original field labels based on multi-modal features by using a field label enhanced memory network; constructing a single-modal and multi-modal processing flow by using a layer-stacking Transformer block, thereby modeling the field dependency relationship in news features in a hierarchical manner while adaptively fusing multi-modal features; introducing a variational mutual information decoupling module to supervise each layer of expert mixed network; obtaining a fake judgment feature by using a field perception Transformer, and outputting a fake judgment result through a classification head. By means of hierarchical field modeling and multi-modal feature fusion, the application effectively solves the distribution difference challenge existing in the multi-field adaptation of social media news.

Description

Technical Field

[0001] This invention belongs to the field of fake news detection technology, specifically involving a multi-domain fake news detection method based on hierarchical latent domain modeling. Background Technology

[0002] Social media has become a key channel for information dissemination. Its multimodal, highly efficient, and flexible nature, while facilitating information exchange, also exacerbates the social harm of fake news. Although numerous studies have focused on automating fake news detection, existing methods are mostly designed for single domains and fail to fully model the multi-domain characteristics of social media news, limiting their detection performance in real-world scenarios. Furthermore, the multimodal nature of news content increases information complexity and dissemination effectiveness, further increasing the difficulty of detection. Therefore, conducting research on fake news detection in multimodal and multi-domain scenarios is of significant practical importance.

[0003] In multi-domain news scenarios, traditional fake news identification methods focused on a single domain, while capable of extracting news sample features beneficial for identifying fake news from different aspects, are susceptible to interference from the distribution differences of samples across different domains. This results in their inability to achieve good performance across all domains of fake news detection tasks. To address this issue, some cross-domain detection methods based on multi-domain datasets have emerged in recent years. Early methods typically used MOE networks to process individual news text content into different domains. However, this approach not only fails to consider the complexity of multi-domain affiliation in real-world news but also cannot adapt to the increasingly complex forms of news content expression. To address this, subsequent multi-domain fake news detection methods construct multi-stream and multi-modal feature processing backbones, thereby integrating multi-domain and multi-modal fake news detection while simultaneously addressing issues such as modal information dependency bias and knowledge dependency bias between different domains. However, while existing multi-domain fake news detection work can handle the problem of uneven distribution across multiple domains relatively well, it still does not explore the relationship between hierarchical semantics and the possible multiple domain affiliation of samples. For news samples taken from social media, samples belonging to a single domain often have potential deep semantic distance relationships with multiple samples belonging to different domains. These relationships are often difficult to capture by single-layer domain processing modules. Therefore, existing anti-spoofing methods ignore the role of hierarchical semantics in domain discrimination during cross-domain anti-spoofing, resulting in their inability to comprehensively consider multiple potential secondary domains associated with the samples, thus revealing their shortcomings. Summary of the Invention

[0004] To address the issues mentioned in the background art, such as ignoring the complexity of existing news sample domains and the difficulty in considering hierarchical semantics for fake news detection, this invention proposes a multi-domain fake news detection method based on hierarchical latent domain modeling, which can complete the forgery detection of news samples from different domains in a multi-domain, multimodal social media fake news environment.

[0005] Technical Solution: To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A multi-domain fake news detection method based on hierarchical latent domain modeling includes the following steps:

[0007] S1: Use a pre-trained feature extractor to obtain multi-view, multi-modal feature embeddings of the news to be detected;

[0008] S2: Enhance the memory network with domain labels and enhance the original domain labels based on multimodal features;

[0009] S3: Constructs single-modal and multimodal processing flows by stacking Transformer blocks, and hierarchically models the domain dependencies in news features while adaptively fusing multimodal features;

[0010] S4: Introduce a variational mutual information decoupling module to supervise each layer of the expert hybrid network;

[0011] S5: Use the domain-aware Transformer to obtain the anti-spoofing features and output the anti-spoofing results through the classification head.

[0012] As a preferred option, the specific implementation details in S1 are as follows:

[0013] For a text With images News samples were used to obtain text embeddings using pre-trained BERT and MAE, respectively. With image embedding ;

[0014] Then, the pre-trained CLIP is used to obtain the aligned text features. Image features Multimodal embeddings are constructed using MLP head mapping. ;

[0015] text With images The single-stream calculation process is as follows:

[0016] ;

[0017] in, and These represent the pre-trained text and image encoders, respectively. and These represent the embedding tensors output after BERT and MAE encoding, respectively;

[0018] The calculation process for CLIP to extract multimodal features is as follows:

[0019] ;

[0020] in, This indicates that aligned text features are obtained using pre-trained CLIP. Aligned image features are obtained using pre-trained CLIP; and These represent the image encoder and text encoder in the CLIP encoder, respectively.

[0021] As a preferred option, the specific implementation details in S2 are as follows:

[0022] S2.1: Construct a memory network consisting of N domain memory blocks, and compute multimodal feature embeddings during the writing process. The similarity score between the content and the corresponding domain memory block is used to generate a weighted erase tensor. With writing tensors Then update the memory blocks of that domain;

[0023] S2.2: During the reading process, multimodal feature embeddings are calculated. Obtain content similarity by comparing the similarity scores with each memory block. This score is then obtained by setting a threshold and a top-k mask, and additional scores are selected based on the most relevant domains. The soft labels are obtained by weighting the one-hot tensor of the original labels. .

[0024] As a preferred option, the specific implementation details in S3 are as follows:

[0025] S3.1: Construct a backbone network with three processing streams;

[0026] Text and image information streams are constructed using stacked ordinary Transformer blocks, while multimodal information streams are constructed using stacked cross Transformer-MOE blocks.

[0027] S3.2: In multimodal flow, based on multimodal features A modal fusion mask is generated and used in the cross-attention blocks for fusing multimodal features with text features and image features, respectively. The outputs of the cross-attention blocks are then concatenated and passed through an FFN feedforward network to obtain the multimodal feature output. ;

[0028] S3.3: Output the multimodal features from S3.2 using a predefined MOE network. Output converted to MOE This enables the extraction of specific domain features at this level.

[0029] As a preferred option, the specific implementation details in S3.2 are as follows:

[0030] The outputs of cross-attention are concatenated and passed through an FFN feedforward network to obtain the multimodal feature output. The specific calculation method is as follows:

[0031] ;

[0032] in, Indicates the first Multimodal features output by the layer-crossing Transformer; This represents cross-attention, which is a hierarchical encoding feature of the text output from the previous layer's single-stream TRF block. Image hierarchical coding features output from the previous single-stream TRF block Multimodal features output by the cross-transformer of the previous layer As input, and based on Network generation via mask Obtain the mask required for attention operations; This indicates a splicing operation between two quantities.

[0033] As a preferred option, the specific implementation details in S3.3 are as follows:

[0034] Overall cross Transformer block The calculation process is as follows:

[0035] ;

[0036] in, Indicates the first Output of layer MOE; and These represent the outputs of static experts and domain experts at the same level, respectively. This represents the supervision loss of the MOE gated network at this layer; This represents the hierarchical encoding features of the text output from the previous single-stream TRF block; This represents the hierarchical coding features of the image output from the previous single-stream TRF block. This represents the multimodal features output by the previous layer's cross-Transformer; This indicates a soft label.

[0037] As a preferred option, the specific implementation details in S4 are as follows:

[0038] First, output the static expert data of the cross Transformer-MOE blocks at the same level. Output with domain experts Let them be denoted as x and y respectively;

[0039] Then, the probability distribution is estimated using variational approximation. and Then, by taking the derivative of the BCE loss formula, we obtain the loss used to maximize the mutual information between the representation and the original data. With the loss used to minimize mutual information between representations ;

[0040] In the calculation of minimizing mutual information loss, cross-entropy loss is used. Maximize representation and joint distribution The conditional probability of random data variables is used to minimize the upper bound of the representation-representation mutual information.

[0041] In the calculation of maximizing mutual information loss, for a discriminative network trained to distinguish between positive sample pairs from the joint distribution and negative sample pairs from the marginal distribution... Use the softplus function Build a and joint distribution of positive samples probability statistics and and Negative sample pair marginal distribution probability statistics This minimizes the binary cross-entropy loss. Equivalent to maximizing the lower bound of JSD mutual information This maximizes the mutual information between the representation and the original data to reduce information loss after decoupling;

[0042] Specifically:

[0043] ;

[0044] ;

[0045] in, and These represent the networks used to distinguish between positive and negative sample pairs. and These represent the mean and standard deviation predicted by the variational encoder, respectively.

[0046] As a preferred option, the specific implementation details in S5 are as follows:

[0047] S5.1: Sum the domain-specific features of the multimodal dataset, the single-stream text features, and the image features, and calculate the gate tensor. Generate classification prediction results for each information stream. ;

[0048] S5.2: Then, DAFormer is used to perform weighted aggregation of the prediction results of each view to obtain the final predicted label. ;

[0049] S5.3: Calculate the total loss function of the model .

[0050] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0051] (1) This invention proposes a multi-domain fake news detection method based on hierarchical latent domain modeling. Compared with other fake news detection methods, while ensuring the extraction of specific features from different original domains, it enables the model to focus on hierarchical semantics and potential related domains, thereby improving the adaptability of the fake news detection to real samples.

[0052] (2) This invention utilizes a layered backbone architecture combining Transformer and MOE, which can perform specific processing of news-related domains while fusing hierarchical multimodal information, ensuring that the model can correctly capture the potential domain associations contained in different levels of semantics.

[0053] (3) This invention uses modular maximum and minimum mutual information decoupling and memory networks to ensure good initialization of MOE domain separation and gating, thereby reducing the model’s misjudgment of different domain affiliations during semantic processing. Attached Figure Description

[0054] Figure 1 This is a flowchart of the multi-domain fake news detection method based on hierarchical latent domain modeling of the present invention;

[0055] Figure 2 This is a detailed architecture diagram of the cross Transformer-MOE block of the present invention;

[0056] Figure 3 This is a schematic diagram of T-SNE class assignment in the Weibo-21 dataset based on the present invention;

[0057] Figure 4 This is the Grad-CAM alignment heatmap of the present invention on the Weibo-21 dataset. Detailed Implementation

[0058] The present invention will be further illustrated below with reference to specific embodiments. These embodiments are implemented based on the technical solutions of the present invention, and it should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0059] like Figures 1-2 As shown, this embodiment provides a multi-domain fake news detection method based on hierarchical latent domain modeling, which specifically includes the following steps:

[0060] S1: Obtain multi-view, multi-modal feature embeddings of the news to be detected using a pre-trained feature extractor. The specific content is as follows:

[0061] For a text With images News samples were used to obtain text embeddings using pre-trained BERT and MAE, respectively. With image embedding Furthermore, aligned text features were obtained using pre-trained CLIP. Image features Multimodal embeddings are constructed using MLP head mapping. .

[0062] text With images The single-stream calculation process is as follows:

[0063]

[0064] in, and These represent the pre-trained text and image encoders, respectively. and These are the embedding tensors output after BERT and MAE encoding, respectively.

[0065] The calculation process for CLIP to extract multimodal features is as follows:

[0066]

[0067] in, This indicates that aligned text features are obtained using pre-trained CLIP. Aligned image features are obtained using pre-trained CLIP; and These represent the image encoder and text encoder in the CLIP encoder, respectively. When they are interconnected, they serve as the input to a multilayer perceptron network (MLP).

[0068] S2: Enhance the memory network with domain labels, and enhance the original domain labels based on multimodal features. The specific content is as follows:

[0069] Construct a memory network consisting of N domain memory blocks, and compute multimodal feature embeddings during the writing process. The similarity score between the content and the corresponding domain memory block is used to generate a weighted erase tensor. With writing tensors Then update the field according to the following formula memory blocks :

[0070]

[0071] in, Hyperparameters for balancing the ratio of erase to write tensors; The original domain labels provided for the dataset.

[0072] During the reading process, multimodal feature embeddings are calculated. Obtain content similarity by comparing the similarity scores with each memory block. This score is then obtained by setting a threshold and a top-k mask, and additional scores are selected based on the most relevant domains. The soft labels are obtained by weighting the one-hot tensor of the original labels with these soft labels. Specifically:

[0073]

[0074] in, For activation functions; Hyperparameters used to balance the proportion of additional scores and original domain labels. The original domain labels provided for the dataset.

[0075] S3: Constructing single-modal and multi-modal processing flows by stacking Transformer blocks, specifically:

[0076] S3.1: Construct a backbone network with three processing streams;

[0077] Text and image information streams are constructed using stacked ordinary Transformer blocks, while multimodal information streams are constructed using stacked cross Transformer-MOE blocks.

[0078] No. Single-stream computation of text and image information in layers, outputting text features. Image feature output The specific calculation methods are as follows:

[0079]

[0080] Here, TRF stands for Transformer block. This represents the hierarchical encoding features of the text output from the previous single-stream TRF block; This represents the hierarchical coding features of the image output from the previous single-stream TRF block.

[0081] S3.2: In multimodal flow, based on multimodal features A modal fusion mask is generated and used in the cross-attention blocks for fusing multimodal features with text features and image features, respectively. The outputs of the cross-attention blocks are then concatenated and passed through an FFN feedforward network to obtain the multimodal feature output. Its overall process consists of... Figure 2 As shown, the specific calculation method is as follows:

[0082]

[0083] in, Indicates the first Multimodal features output by the layer-crossing Transformer; This represents cross-attention, which is a hierarchical encoding feature of the text output from the previous layer's single-stream TRF block. Image hierarchical coding features output from the previous single-stream TRF block Multimodal features output by the cross-transformer of the previous layer As input, and based on Network generation via mask Obtain the mask required for the attention operation; the concatenation operation between two tensors is used here. Operator representation.

[0084] S3.3: Output the multimodal features from the cross-Transformer part in S3.2 using a predefined MOE network. Output converted to MOE This enables the extraction of specific domain features at this level;

[0085] Overall cross Transformer block Architecture such as Figure 2 As shown, the calculation process is as follows:

[0086]

[0087] in, Indicates the first Output of layer MOE; and These represent the outputs of static experts and domain experts at the same level, respectively. This represents the supervision loss of the MOE gated network at this layer; This represents the hierarchical encoding features of the text output from the previous single-stream TRF block; This represents the hierarchical coding features of the image output from the previous single-stream TRF block. This represents the multimodal features output by the previous layer's cross-Transformer; This indicates a soft label.

[0088] S4: Introduce a variational mutual information decoupling module to supervise each layer of the expert hybrid network;

[0089] The specific contents of the variational mutual information decoupling module are as follows:

[0090] Static expert output of the cross Transformer-MOE blocks at the same level Output with domain experts Let x and y be denoted as x and y respectively, then the boundary of mutual information between the two can be defined by formulas such as JSD divergence.

[0091] Specifically, regarding the mutual information between two tensors In this regard, we can refer to the variational information bottleneck theory, based on the methods used in this theory to minimize... The encoder and the one used to maximize The decoders are constructed based on the upper and lower bounds of the solvable mutual information in the data-representation scenario, thereby minimizing the representation. With data Mutual information between And maximize representation With data Mutual information between The upper and lower bounds of its variation are constructed as follows:

[0092] The variational upper bound is:

[0093]

[0094] The variational lower bound is:

[0095]

[0096] in, For data tensors The probability distribution, For data tensors The approximate probability distribution, For data tensors and The joint probability distribution, For data tensors and Marginal probability distribution, To characterize tensors For data tensors The approximate conditional probability distribution; Calculate the KL divergence. and Representing data tensors respectively For the characterization tensor The true conditional probability distribution and its representation tensor For data tensors The true conditional probability distribution; To characterize tensors The probability distribution; For joint distribution The mathematical expectation; For data tensors The mathematical expectation of the true distribution; For joint distribution The mathematical expectation; For data tensors The mathematical expectation of the true distribution; For data The differential entropy.

[0097] In the variational lower bound For data The differential entropy, which in In the case of data, it is a constant that only involves the data generation process.

[0098] In the variational upper bound, due to the marginal distribution It is difficult to solve, so a variational approximation distribution is introduced into the calculation. and through KL divergence constraints .

[0099] The above variational mutual information The calculation of upper and lower bounds is based on or One is external data, and the other is based on representation. And... and When all representations are encoded by a network, due to differential entropy decoder With marginal distribution Since both are unknown, the above formula cannot be directly used to measure the upper and lower bounds of mutual information between two representations. For the upper bound of mutual information between representations, this application can introduce random data variables. ,Will Represented as and Mutual information between them:

[0100]

[0101]

[0102] in, To characterize and and random data Joint mutual information; To characterize and Joint mutual information; for and right The mutual information of the joint conditional distribution; To characterize and random data Joint mutual information; for and right The mutual information of the joint conditional distribution; To characterize and random data Joint mutual information; for and right The mutual information of the joint conditional distribution.

[0103] Then, by combining the representation-data mutual information upper and lower bounds mentioned above, we can obtain the representation-representation mutual information upper bound for minimizing mutual information:

[0104]

[0105] in, The expected value represents the true distribution of the data variable D; For data tensors Approximate conditional probability distribution for the data tensor D; For data tensors The probability distribution, For data tensors Approximate conditional probability distribution for the data tensor D; For data tensors The probability distribution; Represents data tensors For data tensors and The conditional probability of the joint distribution of .

[0106] The latter part is and The conditional probability of the joint distribution of random data variables can be expressed using cross-entropy loss for a discriminative network. Maximize it; for the KL divergence in the first part, since it is obtained through variational approximation at this point... Compared with the original probability distribution Both can be viewed as Gaussian distributions, therefore the KL divergence here has an analytical solution:

[0107]

[0108] in, To characterize tensors For data tensors The approximate posterior distribution; Representation tensor The original probability distribution; This represents a Gaussian distribution of the combination of mean and variance predicted by the variational encoder. Indicates a standard Gaussian distribution; and These represent the mean and standard deviation predicted by the variational encoder, respectively.

[0109] Meanwhile, in the calculation of minimizing mutual information loss, cross-entropy loss is used here. Maximize representation and joint distribution The conditional probability of random data variables is used to minimize the upper bound of the representation-representation mutual information; in the calculation of maximizing the mutual information loss, for a discriminative network trained to distinguish positive sample pairs from the joint distribution and negative sample pairs from the marginal distribution. Use the softplus function Construct a joint positive sample distribution for x and y probability statistics Marginal distribution of negative sample pairs with x and y probability statistics This minimizes the binary cross-entropy loss. Equivalent to maximizing the lower bound of JSD mutual information This maximizes the mutual information between the representation and the original data, thereby reducing information loss after decoupling.

[0110] Therefore, in practical applications, a variational encoder can be used for direct fitting. and This allows for the calculation of the optimizable loss. :

[0111]

[0112] in, This represents the network output used to distinguish positive sample pairs. and These are the mean and standard deviation predicted by the variational encoder, respectively. and They represent the characterization tensors respectively. The distribution means and variance of .

[0113] To maximize the mutual information between representations, the idea of ​​DeepInfoMax is adopted here, based on... and Constructing Jensen-Shannon divergence This allows the estimator to provide gradients that maximize mutual information without needing to compute the exact lower bound of the mutual information:

[0114]

[0115] in, These are negative samples that characterize the marginal distribution. The softplus function is used to determine the network. It is trained to distinguish between positive sample pairs from the joint distribution and negative sample pairs from the marginal distribution. Representation With representation Mutual information between them; and These represent the network outputs used to distinguish between positive and negative sample pairs.

[0116] In practical applications, combining the characteristics of sigmoid and BCE losses, we can deduce that:

[0117]

[0118]

[0119] ;

[0120] in, and These represent the network outputs used to distinguish between positive and negative sample pairs. and These represent the mean and standard deviation predicted by the variational encoder, respectively.

[0121] Minimizing the BCE loss is equivalent to maximizing the lower bound of the JSD mutual information.

[0122] Based on the above derivation, we can obtain the layer The final maximum minimization of mutual information loss This approach aims to separate domain-irrelevant information from domain-specific information while ensuring that the expert output retains its original semantics.

[0123]

[0124] in, This represents the loss that maximizes the mutual information between the representation and the original data; This represents the method used to minimize the loss of mutual information between representations.

[0125] S5: Use the domain-aware Transformer to obtain anti-counterfeiting features and output the anti-counterfeiting results through the classification head;

[0126] The specific content of the aggregated classification results is as follows:

[0127] S5.1: For multimodal domain-specific features, single-stream text features, and image features, sum these three together and calculate the gate tensor. Generate classification prediction results for each information stream;

[0128]

[0129]

[0130] in, The gated tensor projection head corresponding to the i-th classification head; , , All of them are feature tensors output by the stacked Transformers mentioned above; The confidence probability output by the i-th classification head; For obtaining the discrimination confidence level; Represents the feature tensor of the i-th mode; This is the domain-independent feature tensor output by the stacked Transformer above. This is the activation function.

[0131] S5.2: Subsequently, DAFormer (Domain-Aware Transformer) is used to perform weighted aggregation of the prediction results of each view to obtain the final predicted label. ;

[0132] Specifically:

[0133]

[0134]

[0135] in, The gated tensor projection head corresponding to the i-th classification head; , , , All of them are feature tensors output by the stacked Transformers mentioned above; The confidence probability output by the i-th classification head; The decision weights for the i-th classification head output by DAFormer; and These represent the concatenation and addition of tensors, respectively; This represents the final confidence probability for authentication.

[0136] S5.3: Calculate the total loss function of the model ;

[0137] The model's total loss function BCE loss by classification Maximize and minimize mutual information loss and gate loss Together they constitute, specifically:

[0138]

[0139] in, The weights are used to minimize the mutual information loss. The weight for the gating loss.

[0140] The effectiveness and efficiency of the method of the present invention were verified through the following experiments:

[0141] This application uses ACC, F1-score, and AUC as evaluation criteria:

[0142] ACC (Accuracy): A metric that measures the proportion of samples correctly predicted by the model out of the total number of samples, used to evaluate the overall predictive accuracy of the model.

[0143] F1-score: The harmonic mean of precision and recall, which comprehensively reflects the model's performance in detecting fake news on imbalanced datasets, especially its robustness in identifying fake news.

[0144] AUC (Area Under Curve): The area under the ROC curve, used to measure the model's ability to distinguish between true and false news at different thresholds. The higher the value, the better the model's discrimination performance.

[0145] First, the dataset was selected. This invention chose the Weibo dataset and the Weibo-21 dataset: the Weibo dataset contains 7532 training samples, 1000 test samples, and 996 validation samples. This dataset uses 9 domain labels added by existing technology content (MMDFND) to verify multi-domain adaptability.

[0146] The Weibo-21 dataset contains 9128 real-world news samples. This dataset covers nine macro-level news areas: Science, Health, Disease, Entertainment, Society, Military, Politics, Education, and Finance. This application uses rule-based matching to remove invalid links and incorrect labels from the text to improve data quality.

[0147] This application compares the performance of the method of this invention with recent multi-domain fake news detection methods on the Weibo and Weibo-21 datasets. Among the comparison methods, MMOE and MOSE are classic models based on multi-gated expert hybrid models; MDFEND uses a domain-gated network to dynamically adjust the text expert weights, while DAMMFND is an advanced fake news detection scheme based on domain awareness and multi-view discriminators.

[0148] 1. Quantitative experimental results comparing models on different datasets;

[0149] Table 1. Quantitative experimental results of this invention on the Weibo dataset.

[0150]

[0151] Table 2. Quantitative experimental results of this invention on the Weibo-21 dataset.

[0152]

[0153] Based on the results in Tables 1 and 2, it can be concluded that the method in this application outperforms the comparative methods in all three metrics (ACC, F1, AUC) on both the Weibo and Weibo-21 datasets. Notably, even compared to advanced multi-domain detection methods in recent years (such as MMDFND and DAMMFND), the method in this application achieves better results. Experimental results on existing methods show that utilizing domain gating or multi-view features can effectively handle the problem of uneven domain distribution. However, due to the lack of expression of potential multi-domain associations at the deep semantic level in news samples, these methods still have room for improvement in the accuracy of classification boundaries when facing news with complex cross-domain semantics. In contrast, the method in this application performs excellently in both classification accuracy and robustness. This verifies that the layered cross-Transformer-MOE architecture can effectively hierarchically model domain dependencies while integrating modal features; simultaneously, the variational mutual information decoupling module and the domain label-enhanced memory network also play corresponding roles in separating domain-specific knowledge and providing soft domain constraints. For the Weibo-21 dataset, the experimental results of this application consistently demonstrate a significant performance improvement, with ACC and F1 scores reaching 0.9528 and 0.9527 respectively, far exceeding other methods compared.

[0154] Feature distribution and region of interest results under the same evaluation dataset are as follows Figure 3 , Figure 4 As shown, a visual comparison of the method of this application and two typical multi-domain authentication methods is presented.

[0155] Figure 3 In the T-SNE visualization, the results of this application are compared with those of the DAMMFND and MMDFND methods. Clearly, compared to other methods, the method of this application, in terms of generating anti-spoofing features, is able to obtain a more explicit classification boundary while ensuring fewer outliers, thus ensuring more robust performance in multi-domain environments.

[0156] Figure 4 In the Grad-CAM visualization results, the method described in this application performs better on randomly selected samples. However, when dealing with multimodal environments, the comparative methods often produce inaccurate or localized points of interest, leading to less precise text-image alignment. Figure 4The article points out a simple method to speed up your computer: Click "Run" in the Start menu, type "gpedit.ms" and press Enter. A small screen will appear. Then, click "Computer Configuration" -> "Administrative Templates" -> "Network" -> "QoS Data Planner," select "Limit reservable bandwidth," click "Settings," select "Enabled," change the "Bandwidth limit" from 20 to 0, and click "Apply." You can then use 100% of your network speed because Windows XP automatically reserves 20% of the bandwidth. Finally, Figure 4 The paper also demonstrates the results of news semantic detection processing using the method described in this application: the rumor about the XXX event is true.

[0157] Furthermore, the method in this application can more accurately and comprehensively focus on the key image regions corresponding to the text content, achieving a more refined alignment pattern. This demonstrates that the method effectively overcomes the detection difficulty caused by the high complexity of multimodal information through hierarchical processing, improving the consistency and realism of visual effects.

[0158] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multi-domain fake news detection method based on hierarchical latent domain modeling, characterized in that, Includes the following steps: S1: Use a pre-trained feature extractor to obtain multi-view, multi-modal feature embeddings of the news to be detected; S2: Enhance the memory network with domain labels and enhance the original domain labels based on multimodal features; S3: Constructs single-modal and multimodal processing flows by stacking Transformer blocks, and hierarchically models the domain dependencies in news features while adaptively fusing multimodal features; S4: Introduce a variational mutual information decoupling module to supervise each layer of the expert hybrid network; S5: Use the domain-aware Transformer to obtain the anti-spoofing features and output the anti-spoofing results through the classification head.

2. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 1, characterized in that, In S1, the specific implementation details are as follows: For a text With images News samples were used to obtain text embeddings using pre-trained BERT and MAE, respectively. With image embedding ; Then, the pre-trained CLIP is used to obtain the aligned text features. Image features And construct multimodal embeddings through MLP head mapping. ; text With images The single-stream calculation process is as follows: ； in, and These represent the pre-trained text and image encoders, respectively. and These represent the embedding tensors output after BERT and MAE encoding, respectively; The calculation process for CLIP to extract multimodal features is as follows: ； in, This indicates that aligned text features are obtained using pre-trained CLIP. Aligned image features are obtained using pre-trained CLIP; and These represent the image encoder and text encoder in the CLIP encoder, respectively.

3. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 1, characterized in that, In S2, the specific implementation details are as follows: S2.1: Construct a memory network consisting of N domain memory blocks, and compute multimodal feature embeddings during the writing process. The similarity score between the content and the corresponding domain memory block is used to generate a weighted erase tensor. With writing tensors Then update the memory blocks of that domain; S2.2: During the reading process, multimodal feature embeddings are calculated. Obtain content similarity by comparing the similarity scores with each memory block. This score is then obtained by setting a threshold and a top-k mask, and additional scores are selected based on the most relevant domains. The soft labels are obtained by weighting the one-hot tensor of the original labels. .

4. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 1, characterized in that, In S3, the specific implementation details are as follows: S3.1: Construct a backbone network with three processing streams; Text and image information streams are constructed using stacked ordinary Transformer blocks, while multimodal information streams are constructed using stacked cross Transformer-MOE blocks. S3.2: In multimodal flow, based on multimodal features A modal fusion mask is generated and used in the cross-attention blocks for fusing multimodal features with text features and image features, respectively. The outputs of the cross-attention blocks are then concatenated and passed through an FFN feedforward network to obtain the multimodal feature output. ; S3.3: Output the multimodal features from S3.2 using a predefined MOE network. Output converted to MOE This enables the extraction of specific domain features at this level.

5. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 4, characterized in that, In S3.2, the specific implementation details are as follows: The outputs of cross-attention are concatenated and passed through an FFN feedforward network to obtain multimodal feature outputs. The specific calculation method is as follows: ； in, Indicates the first Multimodal features output by the layer-crossing Transformer; This represents cross-attention, which is a hierarchical encoding feature of the text output from the previous layer's single-stream TRF block. Image hierarchical coding features output from the previous single-stream TRF block Multimodal features output by the cross-transformer of the previous layer As input, and based on Network generation via mask Obtain the mask required for attention operations; This indicates a splicing operation between two quantities.

6. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 4, characterized in that, In S3.3, the specific implementation details are as follows: Overall cross Transformer block The calculation process is as follows: ； in, Indicates the first Output of layer MOE; and These represent the outputs of static experts and domain experts at the same level, respectively. This represents the supervision loss of the MOE gated network at this layer; This represents the hierarchical encoding features of the text output from the previous single-stream TRF block; This represents the hierarchical coding features of the image output from the previous single-stream TRF block. This represents the multimodal features output by the previous layer's cross-Transformer; This indicates a soft label.

7. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 1, characterized in that, In S4, the specific implementation details are as follows: First, output the static expert data of the cross Transformer-MOE blocks at the same level. Output with domain experts Let them be denoted as x and y respectively; Then, the probability distribution is estimated using variational approximation. and Then, by taking the derivative of the BCE loss formula, we obtain the loss used to maximize the mutual information between the representation and the original data. With the loss used to minimize mutual information between representations ; In the calculation of minimizing mutual information loss, cross-entropy loss is used. Maximize representation and joint distribution The conditional probability of random data variables is used to minimize the upper bound of the representation-representation mutual information. In the calculation of maximizing mutual information loss, for a discriminative network trained to distinguish between positive sample pairs from the joint distribution and negative sample pairs from the marginal distribution... Use the softplus function Build a and joint distribution of positive samples probability statistics and and Negative sample pair marginal distribution probability statistics This minimizes the binary cross-entropy loss. Equivalent to maximizing the lower bound of JSD mutual information This maximizes the mutual information between the representation and the original data to reduce information loss after decoupling; Specifically: ； ； in, and These represent the networks used to distinguish between positive and negative sample pairs. and These represent the mean and standard deviation predicted by the variational encoder, respectively.

8. The multi-domain fake news detection method based on hierarchical latent domain modeling according to claim 1, characterized in that, In S5, the specific implementation details are as follows: S5.1: Sum the domain-specific features of the multimodal dataset, the single-stream text features, and the image features, and calculate the gate tensor. Generate classification prediction results for each information stream. ; S5.2: Then, DAFormer is used to perform weighted aggregation of the prediction results of each view to obtain the final predicted label. ; S5.3: Calculate the total loss function of the model .

Images