False news detection method based on cross-modal consistency and expert collaborative reasoning
By employing cross-modal consistency and expert collaborative reasoning methods, and utilizing BERT and ResNet-50 to extract features and generate conflict heatmaps, the problem of insufficient text and image inconsistency analysis is solved, achieving high accuracy and interpretability in fake news detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU NORMAL UNIVERSITY
- Filing Date
- 2026-01-15
- Publication Date
- 2026-08-04
AI Technical Summary
Existing fake news detection methods lack effective analytical capabilities when dealing with inconsistencies between text and images, resulting in insufficient recognition accuracy and interpretability.
By employing cross-modal consistency and expert collaborative reasoning methods, text and image features are extracted using the BERT model and ResNet-50, and feature optimization and consistency scoring are performed in conjunction with knowledge graphs to generate conflict heatmaps. Finally, a comprehensive judgment is made through virtual expert collaborative decision-making.
It improves the accuracy and interpretability of fake news detection, accurately identifies false information in news, and provides visual evidence for judgment.
Smart Images
Figure CN122132617B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of natural language processing, computer vision, and artificial intelligence, and more specifically, to a method for detecting fake news based on cross-modal consistency and expert collaborative reasoning. Background Technology
[0002] With the diversification of media content, images and videos play an increasingly important role in news, especially on social media, where they often form a crucial component of news content. However, inconsistencies between images, videos, and text are a common form of fake news. With the rapid development of social media and digital news platforms, the speed and scope of fake news dissemination have increased dramatically, posing a significant negative impact on society. Therefore, it is essential to accurately and quickly identify and prevent the spread of fake news.
[0003] The main purpose of fake news detection is to use artificial intelligence and data analysis technologies to automatically identify and label false information, reduce public exposure to fake news, and thus maintain the authenticity and credibility of information dissemination. Traditional fake news detection methods typically focus on text analysis, relying on keyword matching, grammatical analysis, and other techniques. However, these methods often overlook multimodal information such as images and videos in news reports. Relying solely on textual information for fake news detection often fails to comprehensively capture key information and potential signs of forgery within the news.
[0004] In recent years, with the rapid development of deep learning and multimodal learning, new methods for detecting fake news have emerged that combine multimodal data such as text and images. These methods, by fusing different types of information, can more comprehensively and accurately identify fake news. However, although multimodal detection methods have improved recognition accuracy to some extent, their interpretability and robustness remain significant issues. In particular, when inconsistencies exist between text and images, traditional methods often lack sufficient analytical capabilities to reveal potential conflicts between them.
[0005] Therefore, there is an urgent need for a technical solution to effectively align text and images across modalities and make comprehensive judgments through expert collaborative reasoning, thereby improving the accuracy and interpretability of fake news detection. Summary of the Invention
[0006] To achieve the above objectives, this application provides a fake news detection method based on cross-modal consistency and expert collaborative reasoning, comprising the following steps: Retrieve news content, which includes text and image data, and represent it as follows: ,in, For news text, The data is image data; features are extracted from the news content to obtain text feature vectors. Image feature vectors and knowledge graph vectors; Text feature vectors Image feature vectors Optimize to obtain text embedding vectors and image embedding vector ; Based on text embedding vectors and image embedding vector Perform cross-modal alignment and consistency analysis to generate inconsistency scores. and conflict heatmap; Based on inconsistency scoring, conflict heatmaps, and knowledge graph vectors, expert collaborative reasoning and comprehensive judgment are used to output fake news detection results.
[0007] Feature extraction includes: For news text A pre-trained BERT model is used for contextual semantic encoding to obtain text feature vectors, represented as follows: ,in, It is a sequence of word vectors. Sentence-level text features; For image data Spatial features are extracted using a ResNet-50 convolutional neural network. High-frequency and noise features are extracted from the image through frequency domain analysis to identify key information and potential forgery signs in the image. The resulting image feature vector is: ,in, For global features of the image, It is a set of local features; Entity recognition technology is used to identify key entities and their relationships in text data, generating representative news text. The knowledge graph vector of the context is denoted as: ,in, These are low-dimensional continuous vector representations of the head entity, relation, and tail entity, respectively.
[0008] Optimizing text feature vectors includes the following steps: Word vector sequence The input is modeled using a bidirectional long short-term memory network (Bi-LSTM), represented as follows: ,in, It is the first The forward LSTM hidden state at time step n encodes the sequence from the beginning to the current nth time step. All historical context information up to the word, Then it is the first The inverse LSTM hidden state at time step n encodes the sequence from the end to the current nth time step. All future contextual information up to the word; Word-level representations are obtained by concatenating the two-way hidden states. ; Calculate the weight of each word. By learning the contribution weights of each word to the semantics of the sentence, a global representation at the sentence level can be obtained. , denoted as ,in, , and These are learnable parameters; global representation Sentence-level features According to a fixed gate coefficient We perform weighted summation to obtain the optimized text embedding vector. The calculation formula is as follows: .
[0009] Furthermore, optimizing the image feature vector includes the following steps: Each local feature With global features By stitching the images together and using a fully connected layer to reduce dimensionality, a unified visual representation is obtained. ; The visual representation is divided into regions to obtain the feature set. The result is expressed as: ; Calculate the importance weight of each region The global representation vector of the image is obtained by weighted fusion. , is represented as: ,in, The image region number, and , and These are learnable parameters; vector For optimized image embedding .
[0010] Inconsistency scoring The calculations include: Embed text into vector and image embedding vector Projected onto the same shared semantic space, through a learnable linear mapping matrix Feature transformations are performed on the text and image respectively, as follows: ,in , For the dimensions of the shared semantic space; By applying attention weights to the image with text as a condition, we obtain a text-conditional representation of the image: By using text as a reference to conditionally model the image region, a text-conditional image representation is obtained: ; Based on the region features of the original image Its corresponding text-conditional image features Calculate the inconsistency score .
[0011] Furthermore, based on the original image region features Its corresponding text-conditional image features Calculate the inconsistency score include: For each image region of the image Calculate the features of each region Its textual conditional representation The cosine distance between them is the inconsistency score. : ,in" " denotes the vector dot product, " " denotes the 2-norm of a vector; For all The global cross-modal inconsistency score is obtained by averaging the inconsistency measures of each image region. , is represented as: .
[0012] The original image region features are used as low-pixel image feature maps, and the inconsistency score corresponding to each image region in the image feature map is calculated. The conflict heatmap is generated by mapping back to its spatial location in the original image; the conflict heatmap has region divisions consistent with the original image, based on an inconsistency score. It reflects the discrete conflict intensity distribution, providing interpretability for cross-modal inconsistency analysis results.
[0013] When mapping, the inconsistency score Extended to the original image resolution, including: By applying linear weights in both the horizontal and vertical directions, and based on the relative distances between the target pixel and its four nearest neighbors in the image feature map, the corresponding inconsistency score is determined. A weighted summation is performed to obtain smooth and continuous pixel-level collision intensity values. ; Normalize pixel-level collision intensity values to an interval A color mapping function is introduced to visualize and encode different intensity values, mapping high-contact areas to warm colors and low-contact areas to cool colors, generating a high-resolution heatmap of the conflict.
[0014] Among them, expert collaborative reasoning and comprehensive judgment refer to integrating the output of virtual experts and constructing an expert collaborative reasoning submodule through a gating mechanism to make collaborative decisions; the virtual experts include: consistency analysis experts, knowledge reasoning experts and topic consistency experts. In this case, the consistency analysis directly uses the inconsistency score. As output, knowledge reasoning will use the knowledge graph vector. Input a graph neural network (GNN) for multi-hop inference and output a credibility score based on external knowledge verification. Thematic consistency is derived from text embeddings. and image embedding Extract topic features and calculate cosine similarity as a topic consistency score as output. The gating mechanism concatenates the outputs of the virtual expert into a vector. The weights of each expert opinion are calculated using a trainable gating network to generate weighted fusion features. , is represented as: ; Weighted fusion features Input the Softmax classifier to obtain the predicted probability that the news is fake.
[0015] Furthermore, the fake news detection results include: the confidence level of the output determining the authenticity of the news based on the predicted probability that the news is fake. It combines conflict heatmaps and knowledge graphs to generate interpretable reports.
[0016] This invention accurately identifies false information in news by hierarchically encoding text and images and performing cross-modal alignment judgment. Combined with an expert collaborative reasoning mechanism for comprehensive analysis, it can not only handle the consistency and conflict issues between multimodal data, but also generate conflict heatmaps, providing interpretable judgment basis for the judgment results. This provides a more effective and reliable technical means for the identification and prevention of fake news. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the steps of a fake news detection method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of the fake news detection method provided in the embodiments of the present invention; Figure 3 This is a detailed flowchart illustrating the feature extraction process for news content according to an embodiment of the present invention; Figure 4This is a flowchart of the feature vector optimization process provided according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the cross-modal alignment and consistency analysis process provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the expert collaborative reasoning and comprehensive judgment process provided in the embodiments of the present invention. Detailed Implementation
[0018] The specific implementation of the present invention will now be described in detail with reference to the accompanying drawings.
[0019] The steps of the fake news detection method proposed in this invention are as follows: Figure 1 As shown, it includes the following steps: Step S110: Obtain news content, which includes text data and image data; extract features from the news content to obtain text feature vectors, image feature vectors, and knowledge graph vectors; Specifically, each multimodal news item is represented as: ,in, For news text, For image data; the feature extraction process is as follows: Figure 2 The S210 step is shown below: 1) For the input news text Directly use a pre-trained BERT model for contextual semantic encoding: exist Figure 3 As seen, the text After being input into the BERT model, the text is first encoded by the built-in word-level tokenizer to obtain the corresponding context-aware text representation: , ,in, Indicates the length of the text sequence. The hidden dimension output by BERT. Indicates the first The BERT encoding vectors of individual words or subwords, and the lexical-level text features are represented by word vector sequences as follows: Then on Sentence-level text features are obtained using average pooling: Finally, the text feature vector representation is obtained: .
[0020] 2) Regarding the input image data Spatial features are extracted using a convolutional neural network ResNet-50, and high-frequency and noise features of the image are extracted through frequency domain analysis to identify key information and potential signs of forgery in the image; like Figure 3 As shown, for the image data of the input news Spatial visual features were extracted using a ResNet-50 convolutional neural network pre-trained on the large ImageNet dataset, resulting in high-level feature maps. Recorded as: ,in, The dimension of the output spatial visual semantic feature vector; subsequently, for Global pooling is used to obtain global features of the image. .
[0021] Simultaneously, a visual feature modeling strategy based on frequency domain analysis is introduced to detect whether images have been tampered with. Specifically, image data... Convert to grayscale, perform Fast Fourier Transform (FFT) to obtain the frequency domain representation. A high-pass filter is used. The frequency domain signal is weighted to enhance potential local inconsistencies and anomalous texture information in the high-frequency components. After inverse Fourier transform, the enhanced frequency domain image is obtained, represented as: In this context, ⊙ represents element-wise multiplication. This indicates the amplitude value retrieval operation. Finally, As a single-channel feature map, it is fed into a shallow CNN to extract local representations, and the output feature map is represented as follows: By flattening and arranging the feature vectors of all their spatial locations in order, we obtain the final set of local features, represented as: , , ,in, This indicates that the first term extracted through frequency domain analysis and shallow CNN is... Local regional characteristics, The number of regions is determined by the spatial resolution of the feature maps output by the shallow CNN. The final image feature vector representation is as follows: .
[0022] 3) Then, regarding the text data... Entity recognition technology is used to identify key entities and their relationships in text data to support fact consistency verification; like Figure 3 As shown, this step is achieved through knowledge graphs: using Named Entity Recognition (NER) technology (Stanford CoreNLP) to analyze text data... Key named entities are extracted, including personal names, place names, and organization names. These entities are then linked to the external knowledge graph Wikidata to determine their unique identifiers within Wikidata. For successfully linked entities, one or more triples associated with them in Wikidata are retrieved. Each triple consists of a head entity, a relation, and a tail entity, denoted as . To transform the aforementioned discrete symbolic knowledge into a numerical representation that can be processed by the model, the knowledge graph embedding method TransE is used to vectorize the triples. TransE is based on the translation assumption, that is, triples that satisfy the true relation should approximately satisfy... ,in These are low-dimensional continuous vector representations of the head entity, relation, and tail entity, respectively. Finally, the news text is generated by weighted fusion of the embedding vectors of all relevant triples. The overall knowledge vector of the relevant knowledge context is denoted as: .
[0023] Step S120: Optimize the text feature vector and image feature vector to obtain the text embedding vector and image embedding vector; like Figure 2 As shown in step S220, the optimization process includes: hierarchical encoding of text feature vectors and hierarchical encoding of image feature vectors; see [link to optimization implementation] for details. Figure 4 As shown: 1) When optimizing text feature vectors, feature modeling is performed at the lexical and sentence levels to capture local relationships between words and grammatical and semantic structures in sentences, thereby improving the contextual understanding and semantic representation capabilities of the text; First, the word vector sequence The model is constructed using a bidirectional long short-term memory (Bi-LSTM) network, represented as follows: ,in, It is the first The forward LSTM hidden state at time step n encodes the sequence from the beginning to the current nth time step. All historical context information up to the word, Then it is the first The inverse LSTM hidden state at time step n encodes the sequence from the end to the current nth time step. All future contextual information up to the word; The bidirectional hidden states are then concatenated to obtain a word-level representation. ; To obtain sentence-level global semantic information, a self-attention mechanism is introduced to calculate the weight of each word. By learning the contribution weights of each word to the semantics of the sentence, a global representation at the sentence level can be obtained. : , in, , and These are learnable parameters.
[0024] To fully utilize features at different levels, a manually preset gating coefficient with fixed weights is used to complete the aggregation representation s and sentence-level features. Fusion, based on the semantic characteristics of the target task, determines a fixed scalar value through validation set tuning. This value is used to assign aggregate representations. Sentence-level features The weighting proportions in the fusion result, and then the aggregation representation. Sentence-level features According to a fixed gate coefficient We perform weighted summation to obtain the optimized text embedding vector. The calculation formula is as follows: .
[0025] The final output is the optimized text embedding vector. At the same time, it preserves the word sequence representation of the text. ,in It also encodes the key contextual information and overall semantics of the text.
[0026] 2) When optimizing image feature vectors, a hierarchical self-attention module is constructed. By extracting detailed features of local regions of the image and fusing global information, the overall representation capability of the image is enhanced, ensuring that the global context of the image is fully expressed. Specifically, such as Figure 4 As shown in the diagram, the first step is to unify features and divide regions: each local feature is... With global features The images are stitched together and then subjected to dimensionality reduction through a fully connected layer to obtain a unified visual representation. The visual representation is divided into regions to obtain a feature set. The result is expressed as: Using a self-attention mechanism, the importance weight of each region is calculated. The global representation vector of the image is obtained by weighted fusion. , is represented as: , in, The image region number, and , and These are learnable parameters.
[0027] This vector That is, the optimized image embedding .
[0028] Step S130: Based on the text embedding vector and image embedding vector Perform cross-modal alignment and consistency analysis, including calculating the mutual attention between text and images, generating inconsistency scores and conflict heatmaps; like Figure 2 As shown in step S230, when performing cross-modal alignment and consistency analysis, the text embedding vector is used. and image embedding vector As input, a common attention mechanism and a bidirectional attention mechanism are employed to obtain inconsistency scores and conflict heatmaps. The common attention mechanism is used to capture the correlation between text and images, enhancing the interactivity of text and image features, enabling them to be effectively aligned across modal spaces. The bidirectional attention mechanism is used to further analyze the matching degree and consistency between text and images, improving alignment accuracy through bidirectional interactive analysis and identifying potential inconsistencies or conflicts. Consistency scores are used to quantify the matching degree between text and images, and conflict heatmaps are generated to reflect potential conflict areas between text and images.
[0029] Specific implementation as follows Figure 5 As shown: 1) Embed the text into a vector and image embedding vector Projected onto the same shared semantic space, through a learnable linear mapping matrix Feature transformations are performed on the text and image respectively, as follows: , in ,in, For the dimensions of the shared semantic space; Within this shared semantic space, a correlation matrix S between text and image regions is constructed, denoted as: ,in This is a learnable similarity transformation matrix used to enhance the expressive power of cross-modal feature matching; the elements in the correlation matrix S... Indicates the first The first text word and the first The semantic strength of association between image regions.
[0030] 2) Introduce a bidirectional co-attention mechanism to achieve feature-conditional modeling: On the one hand, attention weighting is applied to images conditioned on text, that is, for The attention weight matrix is obtained by performing Softmax normalization on the image dimension. This leads to the conditional text representation of the image: This representation reflects the most relevant visual information aggregated for each text word under given image semantic conditions; On the other hand, conditional modeling of image regions is performed using text as a reference, that is, for The weight matrix is obtained by performing Softmax normalization on the text dimension. This leads to a text-conditional image representation: This representation depicts the textual semantic information corresponding to each image region under textual semantic constraints.
[0031] The purpose of introducing bidirectional co-attention in this step is to ensure the integrity of the overall cross-modal alignment theoretical framework.
[0032] 3) Based on the original image region features Its corresponding text-conditional image features Calculate the inconsistency score : For each image region of the image ( Calculate the characteristics of each region Its textual conditional representation The cosine distance between them is the inconsistency score. : ,in" " denotes the vector dot product, " "" represents the L2 norm of the vector. This distance metric reflects the degree of deviation between the semantic interpretation of the text and the semantic interpretation of the image region. The larger the value, the stronger the inconsistency between the image region and the text description.
[0033] For all The global cross-modal inconsistency score is obtained by averaging the inconsistency measures of each image region. , is represented as: The inconsistency score, as a whole, characterizes the degree of conflict and inconsistency between the text and image content at the fine-grained semantic level.
[0034] 3) The original image region features can also be used as low-resolution image feature maps for subsequent generation of conflict heatmaps. The inconsistency score corresponding to each image region in the image feature map is then calculated. A conflict heatmap is generated by mapping back to its spatial location in the original image; the conflict heatmap has region divisions consistent with the original image, based on an inconsistency score. Reflecting the discrete conflict intensity distribution can provide interpretability for cross-modal inconsistency analysis results.
[0035] Since region-level features typically correspond to low-resolution feature maps, to obtain pixel-level continuous conflict visualization results, this invention employs a bilinear interpolation upsampling method to map discrete inconsistency scores. Expand to the original image resolution.
[0036] Bilinear interpolation performs linear weighting in both the horizontal and vertical directions, and assigns an inconsistency score based on the relative distance between the target pixel and its four nearest neighbors in the low-resolution collision map. A weighted summation is performed to obtain smooth and continuous pixel-level collision intensity values. .
[0037] Based on this, the pixel-level collision intensity values are normalized to the interval. A color mapping function is introduced to visually encode different intensity values. Using the JET color mapping strategy, high-conflict areas are mapped to warm colors such as red or yellow, while low-conflict areas are mapped to cool colors such as blue or purple, thus generating a high-resolution cross-modal semantic conflict heatmap, represented as: ,in, In a region-level conflict map, pixels The conflict intensity values corresponding to the top left, top right, bottom left, and bottom right of the four nearest grid points at the corresponding location. This represents the normalized distance of the target pixel in the horizontal direction relative to its left neighboring sampling point. This represents the normalized distance of the target pixel relative to its neighboring sampling point in the vertical direction.
[0038] The resulting conflict heatmap is overlaid on the original image, with red highlighted areas visually indicating locations where there are significant inconsistencies between the image content and the textual semantic description. In this way, the model can not only provide overall cross-modal consistency judgment results but also accurately locate potential semantic conflict areas at the spatial level. This transforms the news detection process from a "black box judgment" to "interpretable inference," significantly improving the system's credibility, verifiability, and practical application value in fake news detection tasks.
[0039] Step S140: Based on the inconsistency score, conflict heatmap and knowledge graph vector, perform expert collaborative reasoning and comprehensive judgment, and output the fake news detection result.
[0040] like Figure 2 As shown in step S240, this step uses a gating mechanism combined with knowledge reasoning and consistency analysis to perform comprehensive reasoning, and finally outputs the news authenticity judgment result, confidence analysis report, conflict heat map and knowledge entity reference.
[0041] like Figure 6 As shown, the opinions of three "virtual experts" (consistency analysis expert, knowledge reasoning expert, and topic consistency expert) are integrated, and a collaborative decision-making submodule is constructed through a gating mechanism: 1) Consistency analysis directly uses inconsistency scores. Knowledge reasoning uses knowledge graph vectors Input a graph neural network (GNN) for multi-hop inference and output a credibility score based on external knowledge verification. Thematic consistency is achieved through text embedding. and image embedding Extract topic features and calculate cosine similarity as a topic consistency score, expressed as: ; 2) The gating mechanism concatenates the outputs of the three "virtual experts" into a vector. The weights of each expert opinion are calculated using a trainable gating network, and are expressed as follows: ,in, and These are the parameters of a trainable gating network; Furthermore, generate weighted fusion features. , is represented as: ; 3) Weighted fusion features The predicted probability of a news story being fake is obtained by inputting it into a Softmax classifier, and is represented as: ,in, and These are the classifier parameters.
[0042] The classifier uses the cross-entropy loss function. The Adam optimizer is used to train all parameters in the model end-to-end. All learnable parameters mentioned in this invention are updated through backpropagation. The binary classification cross-entropy loss function is defined as follows: ,in For batch size, For real labels, These are the model's predicted values.
[0043] At the same time, the authenticity is determined and the confidence level is output. The system combines conflict heatmaps and knowledge graph entities to generate interpretability reports as results of fake news detection.
[0044] This invention proposes a fake news detection method based on cross-modal consistency and expert collaborative reasoning. By combining multimodal information from text and images, hierarchical encoding and cross-modal alignment of text and images can accurately identify fake information in news. Furthermore, by combining expert collaborative reasoning mechanism for comprehensive analysis, this method can not only handle the consistency and conflict issues between multimodal data, but also provide interpretable judgment criteria, thus providing a more effective and reliable technical means for the identification and prevention of fake news.
[0045] The above-disclosed embodiments are merely a few specific examples of the present invention. However, the present invention is not limited thereto, and any variations that can be conceived by those skilled in the art should fall within the protection scope of the present invention.
Claims
1. A method for detecting fake news based on cross-modal consistency and expert collaborative reasoning, characterized in that, Includes the following steps: Obtain news content, which includes text data and image data, and is represented as follows: ,in, For news text, The data is image data; features are extracted from the news content to obtain text feature vectors. Image feature vectors and knowledge graph vectors; For the text feature vector Image feature vectors Optimize to obtain text embedding vectors and image embedding vector ; Based on the text embedding vector and the image embedding vector Perform cross-modal alignment and consistency analysis to generate inconsistency scores. and conflict heatmap; Based on the inconsistency score, conflict heatmap and knowledge graph vector, expert collaborative reasoning and comprehensive judgment are performed to output fake news detection results; wherein the inconsistency score comprises: Embedding vectors for text Embedding vectors for images Projecting to the same shared semantic space through a learnable linear mapping matrix Feature transformation for text and images respectively, denoted as: where is the dimension of the shared semantic space; Attention weights the image given the text to get the text-conditioned image representation: ; Condition the image region given the text to get the text-conditioned image representation: ; Based on original image region features Text-conditioned image features corresponding thereto Computing inconsistency scores ; The expert collaborative reasoning and comprehensive judgment refers to integrating the output of virtual experts and constructing an expert collaborative reasoning submodule through a gating mechanism to make collaborative decisions; the virtual experts include: consistency analysis experts, knowledge reasoning experts, and topic consistency experts. In this case, the consistency analysis directly uses the inconsistency score. As output, knowledge reasoning will use knowledge graph vectors. Input a graph neural network (GNN) for multi-hop inference and output a credibility score based on external knowledge verification. Thematic consistency is derived from text embeddings. and image embedding Extract topic features and calculate cosine similarity as the topic consistency score output; The gating mechanism concatenates the virtual expert's output into a vector The weight of each expert opinion is calculated by a trainable gating network to generate a weighted fusion feature The gating mechanism concatenates the virtual expert's output into a vector ; The weighted fused features The input Softmax classifier gets the prediction probability that the news is fake. 2.The fake news detection method of claim 1, wherein, The feature extraction includes: For the news text The pre-trained BERT model is used for context semantic coding to obtain a text feature vector representation as follows: Wherein, is a word vector sequence, is a sentence-level text feature; Regarding the image data Spatial features are extracted using a ResNet-50 convolutional neural network. High-frequency and noise features are extracted through frequency domain analysis to identify key information and potential forgery signs in the image. The resulting image feature vector is: ,in, For global features of the image, It is a set of local features; Entity recognition technology is used to identify key entities and their relationships in text data, generating representative news text. The knowledge graph vector of the context is denoted as: ,in, These are low-dimensional continuous vector representations of the head entity, relation, and tail entity, respectively. 3.The fake news detection method of claim 2, wherein, Optimizing the text feature vector includes the following steps: a sequence of word vectors inputting a bidirectional long short-term memory network to model, denoted as: where, is the forward LSTM hidden state at the th time step, encoding all the historical context information from the beginning of the sequence to the current th word, is the backward LSTM hidden state at the th time step, encoding all the future context information from the end of the sequence to the current th word; Word-level representations are obtained by concatenating the two-way hidden states. ; Calculate the weight of each word. By learning the contribution weights of each word to the semantics of the sentence, a global representation at the sentence level can be obtained. , denoted as ,in, , and These are learnable parameters; global representation Sentence-level features According to a fixed gate coefficient We perform weighted summation to obtain the optimized text embedding vector. The calculation formula is as follows: .
4. The method for detecting fake news according to claim 2, characterized in that, Optimizing the image feature vector includes the following steps: Each local feature With global features By stitching the images together and using a fully connected layer to reduce dimensionality, a unified visual representation is obtained. ; The visual representation is divided into regions to obtain the feature set. Represented as: ; Calculate the importance weight of each region The global representation vector of the image is obtained by weighted fusion. , is represented as: ,in, The image region number, and , and These are learnable parameters; vector For optimized image embedding .
5. The method for detecting fake news according to claim 1, characterized in that, Based on original image region features Its corresponding text-conditional image features Calculate the inconsistency score include: For each image region of the image Calculate the features of each region Its textual conditional representation The cosine distance between them is the inconsistency score. : ,in" " represents the vector dot product," " denotes the 2-norm of a vector; For all The global cross-modal inconsistency score is obtained by averaging the inconsistency measures of each image region. , is represented as: .
6. The method for detecting fake news according to claim 1, characterized in that, The original image region features are used as low-pixel image feature maps, and the inconsistency score corresponding to each image region in the image feature map is calculated. The conflict heatmap is generated by mapping back to its spatial location in the original image; the conflict heatmap has region divisions consistent with the original image, based on the inconsistency score. Reflecting the discrete conflict intensity distribution can provide interpretability for cross-modal inconsistency analysis results.
7. The method for detecting fake news according to claim 6, characterized in that, When performing the mapping, the inconsistency score is... Extended to the original image resolution, including: By applying linear weights in both the horizontal and vertical directions, and based on the relative distances between the target pixel and its four nearest neighbors in the image feature map, the corresponding inconsistency score is determined. A weighted summation is performed to obtain smooth and continuous pixel-level collision intensity values. ; The pixel-level collision intensity values are normalized to an interval. A color mapping function is introduced to visualize and encode different intensity values, mapping high-contact areas to warm colors and low-contact areas to cool colors, generating a high-resolution heatmap of the conflict.
8. The method for detecting fake news according to claim 1, characterized in that, The fake news detection results include: the confidence level of the output determining the authenticity of the news based on the predicted probability that the news is fake. It combines conflict heatmaps and knowledge graphs to generate interpretable reports.