Unified cross-source context enhancement method for multi-source false news detection

Abstract

The invention discloses a unified cross-source context enhancement method for multi-source false news detection, and relates to the field of computer vision, natural language processing and multi-modal information fusion. According to the scheme, rich global context information can be effectively learned and integrated from a plurality of data sources through a unified model, so that the problems that different models need to be trained and the models are poor in performance when the traditional technology faces complex and multi-source data are solved, and the accuracy and reliability of false news detection are remarkably improved. The context enhancement can effectively capture the long-distance interaction among different sources, and can fuse the characteristics of heterogeneous sources, thereby reducing the deviation caused by the difference of data sources. The model can more intelligently align and integrate data from different sources, and the overall performance of false news identification is improved. The local-global double-layer structure improves the ability of the model to process the unseen data source, and improves the stability and detection effect of the system.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision, natural language processing, and multimodal information fusion, and specifically relates to a unified cross-source context enhancement method for multi-source fake news detection. Background Art

[0002] With the rapid development of social media platforms and online news portals, the speed and coverage of information dissemination have reached unprecedented levels. However, this convenience also brings severe challenges. In particular, the widespread dissemination of fake news poses a serious threat to social trust, public opinion, and social stability. Fake news is usually spread through multiple channels, including social networks, instant messaging services, and traditional news websites. This cross-platform dissemination characteristic makes detection methods based on a single platform or a single data type unable to effectively cope with the complexity of multi-source information. Therefore, developing a unified detection framework that can cross multi-source information and identify multimodal fake news has become an urgent research issue to be solved.

[0003] Currently, the research on fake news detection mainly focuses on the analysis of a single source, usually relying on single-modal features such as text, images, or user interaction data to judge the authenticity of news. However, this method ignores the differences between cross-platform contents and is difficult to adapt to the characteristic changes between different platforms in the real world. Single-source models usually lack the ability to process diverse contents in multi-source data scenarios. Although some cross-domain studies have explored feature transfer from the source domain to the target domain, they usually rely on single-source modeling strategies and rarely can effectively combine context information from multiple sources, thus performing poorly in cross-source data scenarios. In addition, there are also challenges in multimodal fusion methods, especially in the semantic differences between visual and text data. This "cross-modal semantic gap" hinders the effective alignment and fusion of information.

[0004] To address these problems, some studies have attempted to improve the generalization ability of the model through cross-domain feature transfer and adaptive methods. For example, Mosallanezhad et al. proposed the REAL-FND framework, which optimizes the representation of news through reinforcement learning to better align features between different domains. In addition, the LIMFA model enhances the adaptability of fake news detection by aligning features from multiple domains. However, these methods mainly target single-domain transfer and cannot cope with the heterogeneity challenges between multi-source data. In addition, many existing methods ignore the unique context differences and changes in the manifestation forms of each platform when dealing with complex multimodal data, which further limits the generalization ability and practicality of the model. Summary of the Invention

[0005] The present invention provides a unified cross-source context enhancement method for multi-source fake news detection. Its core advantage lies in that only one unified model needs to be trained to achieve generalization detection of multiple different sources and various modal data, thus effectively solving the technical problems such as heterogeneity, insufficient context information, inconsistent features, and poor generalization ability in fake news detection in a multi-source data environment in the prior art.

[0006] Based on the above purposes, the technical solution adopted by the present invention is as follows:

[0007] A unified cross-source context enhancement method for multi-source fake news detection includes four modules: multi-source feature encoding, cross-source global context learning, two-level contrast learning, and cross-source multi-modal decoding.

[0008] The multi-source feature encoding module performs intra-source fusion and preliminary characterization on multi-modal features such as text and images through a dedicated encoder and a dynamic time warping strategy, ensuring the consistency of multi-modal features and extracting discriminative information under a single source.

[0009] The cross-source global context learning module, at the global level, through a context enhancement mechanism, merges multi-modal features from different data sources into a unified representation space to capture the long-distance context interaction between multi-modal data at the single-source and cross-source levels, thereby learning more consistent and generalized cross-source features. This context enhancement process effectively synthesizes the differential clues of each data source, can eliminate noise information while strengthening the capture of key features, and finally forms a global feature representation suitable for multi-source fake news detection.

[0010] The two-level contrast learning module performs contrast learning on multi-modal features at two granularities of local and global respectively, aiming to further alleviate the inconsistency problems caused by differences in multi-modal feature spaces from different sources. First, the contrast learning at the local level focuses on feature alignment and difference elimination between modalities such as text and images in a single source, ensuring optimal information fusion and discriminative ability within the same source. Second, the contrast learning at the global level uses the aforementioned global context features to inject key information from other sources in a complementary manner, thus maintaining discriminative consistency and stability at the overall scale.

[0011] The cross-source multi-modal decoding module fuses the final local and global features and classifies them into prediction probabilities.

[0012] The specific steps of this method are as follows:

[0013] Step 1: Multi-source feature encoding;

[0014] Obtain multi-modal data of multiple heterogeneous data sources, including text and image modal data;

[0015] After extracting the text features and image features from the data, the text features and image features are aligned based on the dynamic time warping strategy, and then fused to obtain the local features of each sample in each data source;

[0016] Step 2: Cross-source global context learning;

[0017] First, based on each data source, the local features within the source are weighted and averaged to obtain the source prototype, and then the source prototype is embedded into the local features to obtain the embedded features. At the same time, adversarial training is used to ensure the alignment of features from different data sources;

[0018] Then, based on different data sources, the distances between the embedded features in different data sources are calculated to construct a cross-source sample library;

[0019] Finally, after the embedded features are context-enhanced based on each relevant sample in the cross-source sample library, they are fused with the original embedded features to generate the fused global features;

[0020] Step 3: Two-level contrastive learning;

[0021] Based on a single data source, using the text features as the anchor points, a positive sample set is constructed based on the image features with the same news label, and a negative sample set is constructed based on the image features with different news labels, and the local contrastive loss is calculated;

[0022] Based on all data sources, using the global features of the given sample as the anchor points, a positive sample set is constructed based on the cross-source global features with the same news label, and a negative sample set is constructed based on the cross-source global features with different news labels, and the global contrastive loss is calculated;

[0023] Step 4: Cross-source multimodal decoding;

[0024] First, the local features and global features are concatenated to obtain the fused features;

[0025] Then, a decoder is used to generate the prediction probabilities;

[0026] The classification loss is calculated, the total loss is calculated, and the weights of each loss are adjusted.

[0027] Furthermore, the text features and image features are extracted using pre-trained modality-specific encoders;

[0028] The local features are fused through a cross-attention mechanism and obtained through a non-linear transformation.

[0029] Furthermore, the embedded features are obtained through the following steps:

[0030] B11. For each data source a, the source prototype is calculated by weighted averaging the local features of the text and image samples ;

[0031] B12. For each sample, by using the source prototype as a source - specific prompt to specify its source, the local features of each sample are calculated using the self - attention mechanism with respect to the source prototype for the correlation weights ;

[0032] B13. According to the correlation weights of each sample , the source prototype is dynamically injected into the sample through an adaptive mechanism, and the post - injection feature of the sample with the source - specific prompt embedded is calculated :

[0033] ;

[0034] where, is a tuning parameter, and σ is the sigmoid activation function;

[0035] B14. Through adversarial training, ensure that the post - injection feature is aligned with the distribution of its source. Train the source discriminator to judge the source of the post - injection feature , and calculate the adversarial loss :

[0036] ;

[0037] where, is the source identifier of the i - th sample of data source a, S represents the set of source identifiers, represents the feature of the i - th sample of data source a in the real distribution, and respectively represent taking the expected value under the real distribution and the generated feature distribution , is the cross - entropy loss function.

[0038] Furthermore, the cross - source sample library is obtained in the following way:

[0039] B21. Calculate the cosine similarity between the post - injection feature of the given sample i in the given data source a and the post - injection features of other samples j in other data sources o

[0040] B22. Based on the cosine similarity, select the k most relevant samples from all other data sources to construct the cross - source sample library :

[0041] B23. For other data sources o, the same method is used to obtain the cross-source sample library , where represents other sources in the data source set S except a.

[0042] Further, the context enhancement is achieved through a cross-attention mechanism; then the embedded features are fused with the context-enhanced features, and a normalization layer is used for normalization processing to obtain the fused global features.

[0043] Further, the local contrast loss and the global contrast loss are calculated through the InfoNCE loss function respectively.

[0044] Further, the decoder is the Transformer decoder MMDecoder with a binary classification head.

[0045] Further, the classification loss is the binary cross-entropy loss function;

[0046] The total loss is formulated as follows:

[0047] ;

[0048] where is the classification loss, is the local contrast loss, is the global contrast loss, is the adversarial loss, , , are hyperparameters.

[0049] The beneficial effects of the present invention are:

[0050] (1) Based on the solution of the present invention, the detection model can effectively learn and integrate rich global context information from multiple data sources, thereby overcoming the problem of poor performance of traditional technologies in the face of complex and multi-source data, and significantly improving the accuracy and reliability of fake news detection.

[0051] (2) In the cross-source global context learning module, by introducing context enhancement, the long-distance interaction between different sources can be effectively captured. This method not only processes the context relationship within the same source, but also can fuse the features of heterogeneous sources, reducing the bias caused by data source differences. This strategy enables the model to more intelligently align and integrate data from different sources, improving the overall performance of fake news recognition.

[0052] (3) The two - level contrastive learning module operates at both the local and global levels, enhancing the robustness of the model and its ability to capture details. Local contrastive learning ensures semantic consistency between text and image modalities within a single data source, while global contrastive learning enriches the feature representation of each instance using complementary information from other sources. This two - layer structure improves the model's ability to handle unseen data sources, enhancing the stability and detection performance of the system. Description of the Drawings

[0053] Figure 1 This is the overall flowchart of the model in the embodiment of the present invention.

[0054] Figure 2 This is the flowchart of the multi - source feature encoding module in the embodiment of the present invention.

[0055] Figure 3 This is the flowchart of the cross - source global context learning module in the embodiment of the present invention.

[0056] Figure 4 This is the flowchart of the two - level contrastive learning module in the embodiment of the present invention.

[0057] Figure 5 This is the flowchart of the cross - source multi - modal decoding module in the embodiment of the present invention.

[0058] Figure 6 This is the schematic diagram of the model module structure in the embodiment of the present invention. Detailed Implementation Manner

[0059] The present invention aims to propose a unified cross - source context enhancement method for multi - source fake news detection, solving technical problems such as heterogeneity, insufficient context information, inconsistent features, and poor generalization ability in fake news detection in a multi - source data environment. As Figure 1 and Figure 6 shown, the method includes four modules: multi - source feature encoding, cross - source global context learning, two - level contrastive learning, and cross - source multi - modal decoding.

[0060] During the multi - source feature encoding process, multi - modal data from multiple heterogeneous data sources, including text and image modal data, are obtained. After extracting text features and image features from the data, based on the dynamic time warping strategy, the text features and image features are aligned, and then fused to obtain the local features of each sample in each data source.

[0061] Through a dedicated encoder and the dynamic time warping strategy, in - source fusion and preliminary characterization of multi - modal features such as text and images are performed to ensure the consistency of multi - modal features and extract discriminative information under a single source.

[0062] During the cross - source global context learning process, first, based on each data source, the local features within the source are weighted and averaged to obtain the source prototype, which is used as the prompt information. Then, the source prototype is embedded into the local features to obtain the embedded features. At the same time, adversarial training is adopted to ensure the feature alignment of different data sources. Then, based on different data sources, the distances of the embedded features in different data sources are calculated to construct a cross - source sample library. Finally, after the embedded features are context - enhanced based on each relevant sample in the cross - source sample library, they are fused with the original embedded features to generate the fused global features. This process not only strengthens the capture of key features but also effectively eliminates noise information, improving the overall performance of multi - source fake news detection.

[0063] During the two - level contrast learning process, contrast learning of multi - modal features is carried out at two granularities: local and global, aiming to further alleviate the inconsistency problem caused by the differences in multi - modal feature spaces from different sources. First, based on a single data source, text features are used as the anchor points. A positive sample set is constructed based on the image features with the same news label, and a negative sample set is constructed based on the image features with different news labels. Then, the local contrast loss is calculated. The contrast learning at the local level focuses on the feature alignment and difference elimination between modalities such as text and image within a single source, ensuring optimal information fusion and discriminative ability within the same source. Second, based on all data sources, the global features of a given sample are used as the anchor points. A positive sample set is constructed based on the cross - source global features with the same news label, and a negative sample set is constructed based on the cross - source global features with different news labels. Then, the global contrast loss is calculated. The contrast learning at the global level uses the aforementioned global context features to inject key information from other sources in a complementary manner, thus maintaining the consistency and stability of discrimination at the overall scale.

[0064] In the cross - source multi - modal decoding process, the final local and global features are concatenated and merged, and they are classified into prediction probabilities.

[0065] Embodiment:

[0066] The unified cross - source context enhancement method for multi - source fake news detection in this embodiment includes four parts, which will be specifically described below with reference to the accompanying drawings:

[0067] See Figure 2 , the multi - source feature encoding process includes:

[0068] A1. Obtain multi - modal data from multiple heterogeneous data sources, including text and image modal data;

[0069] A2. Through pre - trained modality - specific encoders respectively from each data source of the text modality and the image modality Extract embedded features , where ;

[0070] A3. Use the dynamic time warping (DTW) algorithm to align the text and image features in each sample:

[0071] In this step, use the dynamic time warping (DTW) algorithm to calculate the distance matrix between the text and image features of each data source each sample , and select the path with the minimum distance through dynamic programming, so as to align the two modal features in the time dimension and obtain the aligned text features and image features , where the dynamic programming process is as follows:

[0072]

[0073] where represents the feature at sequence position p in the text modality and the feature at sequence position q in the image modality The Euclidean distance between them, D(p,q) represents the optimal cumulative distance from sequence (1,1) to (p,q);

[0074] A4. Fusion the text features and image features through the cross-attention mechanism, and perform a non-linear transformation on the fused features to obtain the local features of each sample in each data source : :

[0075] A41. Calculate the similarity between the aligned text features and image features through the cross-attention mechanism to achieve their fusion:

[0076]

[0077] where, , , and are the projection layers for query, key, and value, and d is the embedding dimension.

[0078] A42. Perform a non-linear transformation on the fused features to obtain the final local features , and the calculation formula is:

[0079]

[0080] where, W is the weight matrix of the linear transformation, b is the bias term, and σ is the non-linear activation function.

[0081] See Figure 3 , the cross-source global context learning process includes:

[0082] B1. Calculate the source prototype of each data source by weighted averaging the local features of all samples in each data source , use it as a data source-specific prompt, and dynamically embed the source prototype into the features of each sample using the self-attention mechanism and the adaptive mechanism to obtain the embedded features . At the same time, adopt adversarial training to ensure the feature alignment of different data sources:

[0083] B11. For each data source a, calculate the source prototype by weighted averaging the local features of text and image samples :

[0084]

[0085] where is a learnable weight parameter used to balance the contributions of text and image modalities, and represent the number of text and image modality samples in data source a respectively;

[0086] B12. Each sample specifies its source by using the source prototype as a data source-specific prompt, and calculates the local feature of each sample with respect to the source prototype using the self-attention mechanism :

[0087]

[0088] where, , , and are the weight matrices of query, key, and value, is the local feature of each sample, and d is the feature dimension;

[0089] B13. According to the correlation weight of each sample, dynamically inject the source prototype into the sample through the adaptive mechanism, and calculate the embedded feature with the injected data source-specific prompt:

[0090]

[0091] where, is the adjustment parameter, and σ is the sigmoid activation function.

[0092] B14. Ensuring embedded features through adversarial training Align with the distribution of the source and train the source discriminator Determine the embedded features The source of and calculate the adversarial loss :

[0093]

[0094] in, is the source identifier of the i-th sample of data source a, S represents the source identifier set, represents the characteristics of the i-th sample of data source a in the true distribution, and Respectively represent the real distribution Next, we generate feature distributions Find the expected value below. is the cross entropy loss function.

[0095] B2. Calculate the cosine similarity between the given query sample and samples from other data sources, select the most relevant k samples, and build a cross-source sample library :

[0096] B21. Calculate the embedded features of a given sample i in a given data source a and other data sources o other samples j after embedding The cosine similarity of :

[0097]

[0098] B22. Select the most relevant k samples from all other data sources to build a cross-source sample library :

[0099]

[0100] in, Indicates that the cross-source sample library The elements of The set obtained by cosine similarity with all other samples, is a function that returns the index of the top k most relevant samples based on similarity;

[0101] B23. For other data sources o, use the same method to obtain the cross-source sample library ,in , represents other sources in the data source set S except a.

[0102] B3. Generate enhanced contextual representation by interacting the query sample with each relevant sample in the cross-source sample library through the cross-attention mechanism , and fuse it with the original features to obtain the fused global features :

[0103] B31. Through the cross-attention mechanism, the embedded features of the given sample are interacted with each relevant sample in the cross-source sample library to generate enhanced context representations . The calculation process is as follows:

[0104]

[0105] where is the embedded features of the query sample, and k samples in the cross-source sample library are the keys and values in the cross-attention mechanism, d is the feature dimension, , , and are the weight matrices of the query, key, and value;

[0106] B32. Fuse the context representation with the original features to obtain the fused global features :

[0107]

[0108] where is a normalization layer for normalizing the fused features.

[0109] See Figure 4 , the two-level contrastive learning process includes:

[0110] C1. Local contrastive learning within a single data source:

[0111] C11. Within a single data source, use the text features as the anchor to construct a positive sample set , which includes the image features from the same news label;

[0112] C12. Use the text features as the anchor to construct a negative sample set , which includes the image features from different news labels;

[0113] C13. Calculate the local contrastive loss through the InfoNCE loss function:

[0114]

[0115] Among them, τ is the temperature parameter, which is used to control the sensitivity of the contrastive learning process;

[0116] C2. Global contrastive learning within all data sources:

[0117] C21. Within all data sources, use the global features of the given samples as the anchor points to construct a positive sample set across data sources , which contains cross-source global features from the same news label ;

[0118] C22. Use the global features of the given samples as the anchor points to construct a negative sample set across data sources , which contains cross-source global features from different news labels ;

[0119] C23. Calculate the global contrastive loss through the InfoNCE loss function :

[0120]

[0121] See Figure 5 , the cross-source multimodal decoding process includes:

[0122] D1. Fuse local features and global features:

[0123] In this step, the local features and the global features are fused through a concatenation operation to obtain the fused features :

[0124]

[0125] Among them, represents the concatenation operation, represents the normalization layer.

[0126] D2. Decode the fused features to generate prediction probabilities:

[0127] In this step, use the Transformer decoder MMDecoder with a binary classification head to process the fused features , and generate the prediction probabilities ;

[0128] D3. Calculate the true / false news classification loss:

[0129] In this step, through the binary cross-entropy loss function, calculate the classification loss of each sample based on the true label and the prediction probability ;

[0130] D4. Calculate the total loss:

[0131] In this step, add the adversarial loss from B1, the local contrast loss from C1, the global contrast loss from C2, and the classification loss from D3 to calculate the total loss , and adjust the weights of each loss through hyperparameters , , .

[0132] It can be understood that the present invention is described by some embodiments. Those skilled in the art know that without departing from the spirit and scope of the present invention, various changes or equivalent replacements can be made to these features and embodiments. In addition, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A unified cross - source context enhancement method for multi - source fake news detection, characterized in that, It includes the following steps: Step 1: Multi-source feature encoding; Obtain multi-modal data from multiple heterogeneous data sources, including text and image modal data; After extracting the text features and image features in the data, align the text features and image features based on the dynamic time warping strategy, and then fuse them to obtain the local features of each sample in each data source; Step 2: Cross-source global context learning; First, based on each data source, weight-average the local features within the source to obtain the source prototype, then embed the source prototype into the local features to obtain the embedded features. At the same time, adopt adversarial training to ensure the feature alignment of different data sources; Then, based on different data sources, calculate the distances of the embedded features in different data sources to construct a cross-source sample library; Finally, after the embedded features are context-enhanced based on each relevant sample in the cross-source sample library, they are fused with the original embedded features to generate the fused global features; Step 3: Two-level contrastive learning; Based on a single data source, use the text features as the anchor points, construct a positive sample set based on the image features with the same news label, construct a negative sample set based on the image features with different news labels, and calculate the local contrastive loss; Based on all data sources, use the global features of the given sample as the anchor points, construct a positive sample set based on the cross-source global features with the same news label, construct a negative sample set based on the cross-source global features with different news labels, and calculate the global contrastive loss; Step 4: Cross-source multi-modal decoding; First, concatenate the local features and the global features to obtain the fused features; Then use the decoder to generate the prediction probabilities; Calculate the classification loss, calculate the total loss, and adjust the weights of each loss.

2. The unified cross-source context enhancement method for multi-source fake news detection according to claim 1, wherein The text features and image features are extracted using a pre-trained modality-specific encoder; The local features are fused through a cross-attention mechanism and obtained through a non-linear transformation.

3. A unified cross-source context enhancement method for multi-source fake news detection according to claim 2, characterized in that The embedded features are obtained through the following steps: B11. For each data source a, calculate the source prototype by weighted averaging the local features of the text and image samples ; B12. For each sample, by using the source prototype as the source-specific prompt to specify its source, the local features of each sample are calculated using the self-attention mechanism with respect to the source prototype correlation weights ; B13. According to the correlation weight of each sample , the source prototype is dynamically injected into the sample through an adaptive mechanism , and the embedded features with source-specific prompts are calculated : ； wherein, is a tuning parameter, and σ is the sigmoid activation function; B14. Ensure the alignment of the features after embedding through adversarial training, and train the source discriminator to determine the source of the features after embedding and calculate the adversarial loss :​​ ； Among them, is the source identifier of the i-th sample of data source a, and S represents the set of source identifiers. represents the feature of the i-th sample of data source a in the true distribution. and respectively represent taking the expected value under the true distribution and under the generated feature distribution . is the cross-entropy loss function.

4. A unified cross - source context enhancement method for multi - source fake news detection according to claim 3, characterized in that The cross-source sample library is obtained in the following way: B21. Calculate the embedded features of the given sample i in the given data source a and the embedded features of other samples j in other data sources o for cosine similarity : B22. Select the most relevant k samples from all other data sources based on cosine similarity to construct a cross-source sample library : B23. For other data sources o, the same method is used to obtain the cross-source sample library , where represents other sources in the data source set S except a.

5. A unified cross-source context enhancement method for multi-source fake news detection according to claim 4, characterized in that, The context enhancement is realized through a cross-attention mechanism; then the embedded features are fused with the context-enhanced features, and normalized using a normalization layer to obtain the fused global features.

6. The unified cross-source context enhancement method for multi-source fake news detection according to claim 5, wherein The local contrastive loss and the global contrastive loss are calculated through the InfoNCE loss function respectively.

7. A unified cross-source context enhancement method for multi-source fake news detection according to claim 6, characterized in that The decoder is the Transformer decoder MMDecoder with a binary classification head; 8. A unified cross-source context enhancement method for multi-source fake news detection according to claim 7, characterized in that The classification loss is the binary cross-entropy loss function; The total loss is formulated as follows: ； Among them, is the classification loss, is the local contrast loss, is the global contrast loss, is the adversarial loss, 、 、 are hyperparameters.

Images