A unified cross-source context enhancement method for multi-source fake news detection

Abstract

A unified cross-source context enhancement method for multi-source fake news detection relates to the fields of computer vision, natural language processing, and multimodal information fusion. The solution of the present invention can effectively learn and integrate rich global context information from multiple data sources through a unified model, thereby overcoming the problems of traditional technologies that require training different models and poor model performance when faced with complex, multi-source data, and significantly improving the accuracy and reliability of fake news detection. Context enhancement can effectively capture long-distance interactions between different sources, and can also fuse features from heterogeneous sources to mitigate bias caused by differences in data sources. The model can more intelligently align and integrate data from different sources, improving the overall performance of fake news identification. The local-global two-layer structure improves the model's ability to process unseen data sources, improving the stability of the system and the detection effect.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision, natural language processing and multimodal information fusion, and in particular to a unified cross-source context enhancement method for multi-source fake news detection. Background Art

[0002] With the rapid development of social media platforms and online news portals, the speed and reach of information dissemination have reached unprecedented levels. However, this convenience also brings severe challenges. In particular, the widespread dissemination of false news poses a serious threat to social trust, public opinion, and social stability. False news is often disseminated through multiple channels, including social networks, instant messaging services, and traditional news websites. This cross-platform nature makes detection methods based on a single platform or a single data type ineffective in addressing the complexity of multi-source information. Therefore, developing a unified detection framework that can simultaneously identify multimodal false news across multiple sources has become an urgent research problem.

[0003] Currently, research on fake news detection mainly focuses on the analysis of a single source, usually relying on single-modal features such as text, images, or user interaction data to judge the authenticity of news. However, this approach ignores the differences between cross-platform content and has difficulty adapting to the changes in characteristics between different platforms in the real world. Single-source models generally lack the ability to handle diverse content in multi-source data scenarios. Although some cross-disciplinary studies have explored feature transfer from the source domain to the target domain, they usually rely on single-source modeling strategies and rarely effectively combine contextual information from multiple sources, resulting in poor performance in cross-source data scenarios. In addition, multimodal fusion methods also have challenges, especially in the semantic differences between visual and textual data. This "cross-modal semantic gap" hinders the effective alignment and fusion of information.

[0004] To address these issues, some studies have attempted to improve the generalization ability of models through cross-domain feature transfer and adaptive methods. For example, Mosallanezhad et al. proposed the REAL-FND framework, which optimizes the representation of news through reinforcement learning to better align features across different domains. In addition, the LIMFA model enhances the adaptability of fake news detection by aligning features from multiple domains. However, these methods mainly focus on migration from a single domain and cannot cope with the heterogeneity challenges between multi-source data. In addition, many existing methods ignore the contextual differences and changes in representation specific to each platform when processing complex multimodal data, which further limits the generalization ability and practicality of the model. Summary of the Invention

[0005] The present invention provides a unified cross-source context enhancement method for multi-source fake news detection. Its core advantage is that only one unified model needs to be trained to achieve generalized detection of data from multiple different sources and multiple modalities, thereby effectively solving the technical difficulties of fake news detection in the existing technology in a multi-source data environment, such as heterogeneity, insufficient context information, inconsistent features, and poor generalization ability.

[0006] Based on the above objectives, the technical solutions adopted by the present invention are as follows:

[0007] A unified cross-source context enhancement method for multi-source fake news detection includes four modules: multi-source feature encoding, cross-source global context learning, two-level contrastive learning, and cross-source multimodal decoding.

[0008] The multi-source feature encoding module uses a dedicated encoder and dynamic time warping strategy to perform intra-source fusion and preliminary characterization of multimodal features such as text and images, ensuring the consistency of multimodal features and extracting information with discriminative significance from a single source.

[0009] The cross-source global context learning module uses a context enhancement mechanism at the global level to merge multimodal features from different data sources into a unified representation space. This allows the module to capture the long-range contextual interactions between multimodal data within a single source and across sources, thereby learning more consistent and generalizable cross-source features. This context enhancement process effectively integrates differentiated cues from each data source, eliminating noise while enhancing the capture of key features, ultimately forming a global feature representation suitable for multi-source fake news detection.

[0010] The two-level contrastive learning module performs contrastive learning on multimodal features at both local and global granularities, aiming to further alleviate the inconsistency problem caused by differences in multimodal feature spaces across different sources. First, contrastive learning at the local level focuses on feature alignment and difference elimination between modalities such as text and images in a single source, ensuring optimal information fusion and discrimination capabilities within the same source. Second, contrastive learning at the global level utilizes the aforementioned global contextual features to complementarily inject key information from other sources, thereby maintaining consistency and stability in discrimination at the overall scale.

[0011] The cross-source multimodal decoding module fuses the final local and global features and classifies them into prediction probabilities.

[0012] The specific steps of this method are as follows:

[0013] Step 1: Multi-source feature encoding;

[0014] Acquire multimodal data from multiple heterogeneous data sources, including text and image modal data;

[0015] After extracting text features and image features from the data, the text features and image features are aligned based on the dynamic time warping strategy, and then fused to obtain the local features of each sample in each data source;

[0016] Step 2: Cross-source global context learning;

[0017] First, for each data source, the local features within the source are weighted averaged to obtain the source prototype. The source prototype is then embedded into the local features to obtain the embedded features. At the same time, adversarial training is used to ensure feature alignment between different data sources.

[0018] Then, based on different data sources, the distance between the embedded features in different data sources is calculated to build a cross-source sample library;

[0019] Finally, the embedded features are contextually enhanced based on each relevant sample in the cross-source sample library and fused with the original embedded features to generate the fused global features.

[0020] Step 3: Two-level contrastive learning;

[0021] Based on a single data source, using text features as anchors, constructing a positive sample set based on image features of the same news label, and a negative sample set based on image features of different news labels, and calculating the local contrast loss;

[0022] Based on all data sources, we use the global features of a given sample as anchors, construct a positive sample set based on the cross-source global features of the same news label, and a negative sample set based on the cross-source global features of different news labels, and calculate the global contrast loss.

[0023] Step 4: Cross-source multimodal decoding;

[0024] First, local features and global features are spliced ​​to obtain fused features;

[0025] The decoder is then used to generate predicted probabilities;

[0026] Calculate the classification loss, calculate the total loss and adjust the weights of each loss.

[0027] Furthermore, the text features and image features are extracted using a pre-trained modality-specific encoder;

[0028] The local features are fused through a cross-attention mechanism and obtained through nonlinear transformation.

[0029] Furthermore, the embedded features are obtained by the following steps:

[0030] B11. For each data source a, calculate the source prototype by taking a weighted average of the local features of the text and image samples ;

[0031] B12. Each sample is processed by the source prototype As source-specific hints to specify its origin, a self-attention mechanism is used to compute local features for each sample. Relative to the source prototype The relevance weight ;

[0032] B13. According to the relevance weight of each sample , dynamically converting the source prototype into In the injected sample, the embedded features of the injection source specific prompts are calculated :

[0033] ;

[0034] in, is the adjustment parameter, σ is the sigmoid activation function;

[0035] B14. Ensure embedded features through adversarial training Align with the distribution of the source and train the source discriminator Determine the embedded features The source of and calculate the adversarial loss :

[0036] ;

[0037] in, is the source identifier of the i-th sample of data source a, S represents the source identifier set, represents the characteristics of the i-th sample of data source a in the true distribution, and Represents the real distribution Next and generate feature distribution Find the expected value below. is the cross entropy loss function.

[0038] Furthermore, the cross-source sample library is obtained in the following manner:

[0039] B21. Calculate the embedded features of a given sample i in a given data source a and other data sources o and other sample j after embedding features Cosine similarity of :

[0040] B22. Based on cosine similarity, select the most relevant k samples from all other data sources to build a cross-source sample library :

[0041] B23. For other data sources o, use the same method to obtain the cross-source sample library ,in , represents other sources in the data source set S except a.

[0042] Furthermore, the context enhancement is achieved through a cross-attention mechanism; then the embedded features are fused with the context-enhanced features, and normalized using a normalization layer to obtain the fused global features.

[0043] Furthermore, the local contrast loss and the global contrast loss are respectively calculated by the InfoNCE loss function.

[0044] Furthermore, the decoder is a Transformer decoder MMDecoder with a binary classification head.

[0045] Furthermore, the classification loss is a binary cross entropy loss function;

[0046] The total loss is formulated as follows:

[0047] ;

[0048] in, is the classification loss, is the local contrast loss, is the global contrast loss, To combat losses, 、 、 is a hyperparameter.

[0049] The beneficial effects of the present invention are:

[0050] (1) Based on the solution of the present invention, the detection model can effectively learn and integrate rich global context information from multiple data sources, thereby overcoming the problem that traditional technologies perform poorly when faced with complex, multi-source data, and significantly improving the accuracy and reliability of fake news detection.

[0051] (2) In the cross-source global context learning module, we introduce context enhancement to effectively capture long-range interactions between different sources. This approach not only handles contextual relationships within the same source, but also integrates features from heterogeneous sources, mitigating bias caused by differences in data sources. This strategy enables the model to more intelligently align and integrate data from different sources, improving the overall performance of fake news identification.

[0052] (3) The two-level contrastive learning module operates at both local and global levels, enhancing the model’s robustness and detail-capturing capabilities. Local contrastive learning ensures semantic consistency between text and image modalities within a single data source, while global contrastive learning enriches the feature representation of each instance using complementary information from other sources. This two-layer structure improves the model’s ability to handle unseen data sources, enhancing the system’s stability and detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is an overall flow chart of the model in an embodiment of the present invention.

[0054] Figure 2 4 is a flow chart of a multi-source feature encoding module in an embodiment of the present invention.

[0055] Figure 3 This is a flow chart of the cross-source global context learning module in an embodiment of the present invention.

[0056] Figure 4 4 is a flow chart of a two-level comparative learning module in an embodiment of the present invention.

[0057] Figure 5 This is a flow chart of a cross-source multimodal decoding module in an embodiment of the present invention.

[0058] Figure 6 Schematic diagram of the model module structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] This paper aims to propose a unified cross-source context enhancement method for multi-source fake news detection, which can solve the technical problems of fake news detection in the prior art, such as heterogeneity, insufficient context information, inconsistent features and poor generalization ability in multi-source data environment. Figure 1 and Figure 6 As shown in the figure, the method includes four modules: multi-source feature encoding, cross-source global context learning, two-level contrastive learning, and cross-source multimodal decoding.

[0060] In the multi-source feature encoding process, multimodal data from multiple heterogeneous data sources are obtained, including text and image modal data; after extracting the text features and image features in the data, the text features and image features are aligned based on the dynamic time warping strategy, and then fused to obtain the local features of each sample in each data source.

[0061] Through dedicated encoders and dynamic time warping strategies, multimodal features such as text and images are fused and preliminarily represented within the source to ensure the consistency of multimodal features and extract discriminative information from a single source.

[0062] During cross-source global context learning, the team first performs a weighted average of local features within each data source to obtain a source prototype, which serves as a prompt. The source prototype is then embedded into the local features to generate an embedded feature, while adversarial training is employed to ensure feature alignment across different data sources. The distances between the embedded features across different data sources are then calculated to construct a cross-source sample library. Finally, the embedded features are contextually enhanced based on each relevant sample in the cross-source sample library and then fused with the original embedded features to generate a fused global feature. This process not only enhances the capture of key features but also effectively eliminates noise, improving the overall performance of multi-source fake news detection.

[0063] In the two-level contrastive learning process, multimodal features are contrastively learned at both local and global granularities, aiming to further alleviate the inconsistency problem caused by differences in multimodal feature spaces across different sources. First, based on a single data source, using text features as anchors, a positive sample set is constructed based on image features with the same news label, and a negative sample set is constructed based on image features with different news labels, and a local contrastive loss is calculated. Local-level contrastive learning focuses on aligning and eliminating differences between modalities such as text and images in a single source, ensuring optimal information fusion and discrimination within the same source. Second, based on all data sources, using the global features of a given sample as anchors, a positive sample set is constructed based on cross-source global features with the same news label, and a negative sample set is constructed based on cross-source global features with different news labels, and a global contrastive loss is calculated. Global-level contrastive learning utilizes the aforementioned global contextual features to complementarily inject key information from other sources, thereby maintaining consistency and stability of discrimination across the entire scale.

[0064] The cross-source multimodal decoding process combines the final local and global features by splicing and classifying them into prediction probabilities.

[0065] Example:

[0066] The unified cross-source context enhancement method for multi-source fake news detection in this embodiment includes four parts, which are described in detail below with reference to the accompanying drawings:

[0067] See also Figure 2 ,The multi-source feature encoding process includes:

[0068] A1. Acquire multimodal data from multiple heterogeneous data sources, including text and image modal data;

[0069] A2. Modality-specific encoders through pre-training From each data source Text modal and image modality Extract embedded features ,in ;

[0070] A3. Use the Dynamic Time Warping (DTW) algorithm to align the text and image features in each sample:

[0071] In this step, the Dynamic Time Warping (DTW) algorithm is used to calculate the Each sample The distance matrix between the text and image features is obtained, and the path with the minimum distance is selected through dynamic programming to align the two modal features in the time dimension to obtain the aligned text features. and image features , where the dynamic programming process is as follows:

[0072]

[0073] in Represents the feature at sequence position p in the text modality and the feature at sequence position q in the image modality The Euclidean distance between them, D(p,q) represents the optimal cumulative distance from the sequence (1,1) to (p,q);

[0074] A4. Fuse text features and image features through the cross attention mechanism, and perform Perform nonlinear transformation to obtain each data source Each sample Local features :

[0075] A41. Calculate aligned text features through the cross-attention mechanism and image features The similarity between them is used to achieve the fusion of the two:

[0076]

[0077] in, , ,and is the projection layer for query, key and value, and d is the embedding dimension.

[0078] A42. Perform nonlinear transformation on the fused features to obtain the final local features , the calculation formula is:

[0079]

[0080] Among them, W is the weight matrix of the linear transformation, b is the bias term, and σ is the nonlinear activation function.

[0081] See also Figure 3 ,The cross-source global context learning process includes:

[0082] B1. Calculate the source prototype of each data source by taking a weighted average of the local features of all samples in each data source , as a data source specific hint, and dynamically convert the source prototype into Embedded into the features of each sample to obtain the embedded features , while using adversarial training to ensure feature alignment across different data sources:

[0083] B11. For each data source a, calculate the source prototype by taking a weighted average of the local features of the text and image samples :

[0084]

[0085] in is a learnable weight parameter used to balance the contributions of text and image modalities, and Represents the number of samples of text and image modalities in data source a respectively;

[0086] B12. Each sample is processed by the source prototype As source-specific hints to specify its origin, a self-attention mechanism is used to compute local features for each sample. Relative to the source prototype The relevance weight :

[0087]

[0088] in, , ,and is the weight matrix of query, key and value, is the local feature of each sample, and d is the feature dimension;

[0089] B13. According to the relevance weight of each sample , dynamically converting the source prototype into In the injected sample, the embedded features of the injection source specific prompts are calculated :

[0090]

[0091] in, is the adjustment parameter and σ is the sigmoid activation function.

[0092] B14. Ensure embedded features through adversarial training Align with the distribution of the source and train the source discriminator Determine the embedded features The source of and calculate the adversarial loss :

[0093]

[0094] in, is the source identifier of the i-th sample of data source a, S represents the source identifier set, represents the characteristics of the i-th sample of data source a in the true distribution, and Represents the real distribution Next and generate feature distribution Find the expected value below. is the cross entropy loss function.

[0095] B2. Calculate the cosine similarity between the given query sample and samples from other data sources, select the most relevant k samples, and build a cross-source sample library :

[0096] B21. Calculate the embedded features of a given sample i in a given data source a and other data sources o and other sample j after embedding features Cosine similarity of :

[0097]

[0098] B22. Select the most relevant k samples from all other data sources to build a cross-source sample library :

[0099]

[0100] in, Indicates the cross-source sample library Elements of , representing calculations The set obtained by cosine similarity with all other samples, is a function that returns the indices of the top k most relevant samples based on similarity;

[0101] B23. For other data sources o, use the same method to obtain the cross-source sample library ,in , represents other sources in the data source set S except a.

[0102] B3. Generate enhanced contextual representation by interacting the query sample with each relevant sample in the cross-source sample library through the cross-attention mechanism , and fuse it with the original features to obtain the fused global features :

[0103] B31. Through the cross attention mechanism, the embedded features of the given sample are Cross-source sample library Each relevant sample in interacts to generate enhanced contextual representation The calculation process is as follows:

[0104]

[0105] in, It is the embedded feature of the query sample, cross-source sample library The k samples in are the keys and values ​​in the cross attention mechanism, d is the feature dimension, 、 ,and is the weight matrix of query, key and value;

[0106] B32. Representing the context Fuse with the original features to obtain the fused global features :

[0107]

[0108] in, It is a normalization layer used to standardize the fused features.

[0109] See also Figure 4 , the two-level contrastive learning process includes:

[0110] C1. Local contrastive learning within a single data source:

[0111] C11. Use text features within a single data source As an anchor point, construct a positive sample set , which includes image features from the same news tag ;

[0112] C12. Using text features As an anchor point, construct a negative sample set , which includes image features from different news tags ;

[0113] C13. Calculate local contrast loss through InfoNCE loss function :

[0114]

[0115] Where τ is the temperature parameter used to control the sensitivity of the contrastive learning process;

[0116] C2. Global contrastive learning across all data sources:

[0117] C21. Use global features of a given sample across all data sources As an anchor point, build a positive sample set across data sources , which contains cross-source global features from the same news tag ;

[0118] C22. Using global features of a given sample As an anchor point, construct a negative sample set across data sources , which contains cross-source global features from different news tags ;

[0119] C23. Calculate global contrast loss through InfoNCE loss function :

[0120]

[0121] See also Figure 5 , the cross-source multimodal decoding process includes:

[0122] D1. Fusion of local features and global features:

[0123] In this step, local features are stitched together and global features Perform fusion to obtain the fused features :

[0124]

[0125] in, Represents a splicing operation, represents a normalization layer.

[0126] D2. Decode the fused features and generate prediction probabilities:

[0127] In this step, the fused features are decoded using the Transformer decoder MMDecoder with a binary classification head. To process, , generating predicted probabilities ;

[0128] D3. Calculate the true and false news classification loss:

[0129] In this step, the classification loss of each sample is calculated based on the true label and predicted probability through the binary cross entropy loss function. ;

[0130] D4. Calculate the total loss:

[0131] In this step, the adversarial loss from B1, the local contrast loss from C1, the global contrast loss from C2, and the classification loss from D3 are added together to calculate the total loss , and through the hyperparameters 、 、 Adjust the weight of each loss.

[0132] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A unified cross-source context enhancement method for multi-source fake news detection, characterized by: The following steps are involved: Step 1: Multi-source feature encoding; Acquire multimodal data from multiple heterogeneous data sources, including text and image modal data; After extracting text features and image features from the data, the text features and image features are aligned based on the dynamic time warping strategy, and then fused to obtain the local features of each sample in each data source; Step 2: Cross-source global context learning; First, for each data source, the local features within the source are weighted averaged to obtain the source prototype. The source prototype is then embedded into the local features to obtain the embedded features. At the same time, adversarial training is used to ensure feature alignment between different data sources. Then, based on different data sources, the distance between the embedded features in different data sources is calculated to build a cross-source sample library; Finally, the embedded features are contextually enhanced based on each relevant sample in the cross-source sample library and fused with the original embedded features to generate the fused global features. Step 3: Two-level contrastive learning; Based on a single data source, using text features as anchors, constructing a positive sample set based on image features of the same news label, and a negative sample set based on image features of different news labels, and calculating the local contrast loss; Based on all data sources, we use the global features of a given sample as anchors, construct a positive sample set based on the cross-source global features of the same news label, and a negative sample set based on the cross-source global features of different news labels, and calculate the global contrast loss. Step 4: Cross-source multimodal decoding; First, local features and global features are spliced ​​to obtain fused features; The decoder is then used to generate predicted probabilities; Calculate the classification loss, calculate the total loss and adjust the weights of each loss.

2. A unified cross-source context enhancement method for multi-source fake news detection according to claim 1, characterized in that: The text features and image features are extracted using a pre-trained modality-specific encoder; The local features are fused through a cross-attention mechanism and obtained through nonlinear transformation.

3. A unified cross-source context enhancement method for multi-source fake news detection according to claim 2, characterized in that: The embedded features are obtained by the following steps: B11. For each data source a, calculate the source prototype by taking a weighted average of the local features of the text and image samples ; B12. Each sample is processed by the source prototype As source-specific hints to specify its origin, a self-attention mechanism is used to compute local features for each sample. Relative to the source prototype The relevance weight ; B13. According to the relevance weight of each sample , dynamically converting the source prototype into In the injected sample, the embedded features of the injection source specific prompts are calculated : ； in, is the adjustment parameter, σ is the sigmoid activation function; B14. Ensure embedded features through adversarial training Align with the distribution of the source and train the source discriminator Determine the embedded features The source of and calculate the adversarial loss : ； in, is the source identifier of the i-th sample of data source a, S represents the source identifier set, represents the characteristics of the i-th sample of data source a in the true distribution, and Represents the real distribution Next and generate feature distribution Find the expected value below. is the cross entropy loss function.

4. A unified cross-source context enhancement method for multi-source fake news detection according to claim 3, characterized in that: The cross-source sample library is obtained in the following way: B21. Calculate the embedded features of a given sample i in a given data source a and other data sources o and other sample j after embedding features Cosine similarity of : B22. Based on cosine similarity, select the most relevant k samples from all other data sources to build a cross-source sample library : B23. For other data sources o, use the same method to obtain the cross-source sample library ,in , represents other sources in the data source set S except a.

5. A unified cross-source context enhancement method for multi-source fake news detection according to claim 4, characterized in that: The context enhancement is achieved through a cross-attention mechanism; the embedded features are then fused with the context-enhanced features, and normalized using a normalization layer to obtain a fused global feature.

6. A unified cross-source context enhancement method for multi-source fake news detection according to claim 5, characterized in that: The local contrast loss and the global contrast loss are respectively calculated by the InfoNCE loss function.

7. A unified cross-source context enhancement method for multi-source fake news detection according to claim 6, characterized in that: The decoder is a Transformer decoder MMDecoder with a binary classification head.

8. A unified cross-source context enhancement method for multi-source fake news detection according to claim 7, characterized in that: The classification loss is a binary cross entropy loss function; The total loss is formulated as follows: ； in, is the classification loss, is the local contrast loss, is the global contrast loss, To combat losses, 、 、 is a hyperparameter.

Images