A fake news detection method based on temporal aggregation multimodal graph neural network
Through the method of aggregating multimodal graph neural network based on time sequence, multimodal features of news are extracted and aggregated, and event-level news clusters are constructed, which solves the problem of news detection under the same topic in the existing technology, and realizes high-precision fake news detection , improves the interaction effect of multimodal information.
Patent Information
- Application Number
- CN202311234209.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-22
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-09-22
AI Technical Summary
Existing fake news detection technology is difficult to accurately detect the credibility of similar news under the same topic, ignoring the complementary between logical conflicts in time and semantic information, and the global correlation capture effect of multimodal features is poor.
Using the method of aggregating multimodal graph neural network based on time sequence, we can extract the text, pictures and external knowledge features of news to form a multimodal subgraph with semantic complementarity, and aggregate posts with near time and the same topic to build an event-level News cluster, a graph neural network is used to perform double-layer detection of post-level and event-level news.
It realizes high-precision fake news detection, improves the multimodal information interaction and correlation capture effect, and improves the accuracy of fake news detection and model generalization capabilities.
Smart Images

Figure CN117290504B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of fake news detection, and specifically relates to a fake news detection method based on a time-series aggregation multimodal graph neural network. Background Art
[0002] In recent years, the problem of fake news on social media has become increasingly serious, bringing huge impacts and hidden dangers. The inflammatory information contained in fake news can, under certain conditions, intensify certain social relations, increase social divisions, and undermine social harmony. The existence of a large number of fake news will make the public question the authenticity and accuracy of information, thereby reducing the credibility of media information. In the economic field, some important decisions rely on public opinion or judgments on the current situation. Fake news will mislead these judgments and decisions, and then cause substantial losses to the economic field. Therefore, the development of efficient and accurate fake news automatic detection tools has become an important topic and research direction in the field of information technology. According to the different modalities, the existing models can be roughly summarized into two categories: fake news detection based on single modality and multimodality. Fake news detection based on single modality focuses on the text of the news, mainly extracting features manually or combining machine learning. Fake news detection based on multimodality is not limited to the news text itself, but also includes pictures in the news and external background knowledge, and most of them are combined with deep learning methods.
[0003] Early work on single-modal fake news detection focused on modeling and classification of news text content, including regular features such as word frequency and part of speech extracted manually. After the features are extracted, they can be classified in combination with traditional machine learning models. H. Rashkin, E. Choi, Y. J. Jin, S. Volkova, and Y. Choi analyzed the language features of fake text in their paper "Truth of varying shades: Analyzing language in fake news and political fact-checking" (Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 2931-2937, 2017), and effectively understood the difference between reliable and unreliable news sources. However, this method relies on features designed by domain experts and has poor generalization ability. Moreover, these methods that rely only on text content face certain limitations. Many fake news are combined with image or video content, which cannot be judged by simple text classification. Secondly, external knowledge also provides important contextual information. The content of some fake news may contradict real facts or common sense, which requires the model to have strong semantic understanding and reasoning capabilities. In their paper "Fake news detection via knowledge-driven multimodal graph convolutional networks" (Proceedings of the 2020 international conference on multimedia retrieval, pp. 540-547, 2020), Y. Wang, S. Qian, J. Hu, Q. Fang, and C. Xu jointly modeled text information, knowledge concepts, and visual information into a unified framework KMGCN for multimodal fake news detection. Experiments have shown that the introduction of multimodal information can achieve better classification performance than deep learning methods that only use single-modal information. However, the graph convolutional network (GCN) used in this method can only consider the adjacent relationship of nodes within a few hops, and cannot capture longer-range non-local connections, which is not very conducive to obtaining the global correlation of multimodal features. The patent application “Fake news detection method based on generative adversarial network and multimodal fusion” with publication number CN113221872B discloses a fake news detection method based on generative adversarial network and multimodal fusion.This method can extract the text features and image features of news, and input the multimodal features into the neural network through tensor fusion to obtain the classification results of news authenticity, solving the problem that the model training process is limited to a single data set, resulting in low fake news detection accuracy and model generalization. This method requires a data set that contains pictures and words describing pictures, but high-quality annotated data is rare, which limits the training effect of the model. In addition, the above model is limited to the detection of post-level news, ignoring the temporal connection between news on the same topic. News on the same topic at adjacent times often have high similarities, and can complement each other at the semantic level.
[0004] The above-mentioned fake news detection methods mainly focus on the text features themselves or the multimodal information fusion representation of a single news post, distinguishing the language style and form of fake news, the inconsistency between news text and image content, and the fiction of background relationship knowledge in news. However, in the face of the current situation where fake news and real news are mixed, and a large number of similar posts are published at the same time, the logical conflicts in time between news events under the same topic are ignored, and it is difficult to detect the credibility of similar news under the same topic, ignoring the mutual complementation of information at the semantic level. Therefore, it is urgent to develop a new fake news detection solution. Summary of the invention
[0005] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned prior art and provide a fake news detection method based on a time-series aggregation multimodal graph neural network. By fusing the features of text information, image information and external knowledge, a multimodal subgraph with semantic complementarity enhancement is formed, and post-level news with similar time and the same theme are aggregated to form an event-level news cluster. The graph neural network is used to learn the features within and between posts to achieve high-precision fake news detection and classification effects. Double-layer detection of post-level and event-level news can be performed, solving the problem that the existing fake news detection technology only detects a single post-level news, but is difficult to accurately detect event-level news composed of news with the same theme at adjacent times. The interaction and correlation capture effect between multimodalities is improved by constructing a multimodal graph, thereby improving the fake news detection performance.
[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:
[0007] A false news detection method based on a time series aggregation multimodal graph neural network uses a pre-trained false news detection model based on a time series aggregation multimodal graph neural network to detect false news. The false news detection process of the model includes:
[0008] Step 1: extract news multimodal information features, convert feature entities into vectors, and obtain news subgraphs represented by vectors;
[0009] Step 2: Aggregate multimodal information features based on the news subgraph represented by the vector to obtain a multimodal subgraph of the news post;
[0010] Step 3: Construct event-level news, i.e., event clusters, based on the multimodal subgraph of news posts;
[0011] Step 4: The post-level news detection module performs post-level news detection on the multimodal subgraph of the news post to obtain a post-level output;
[0012] Step 5: The event-level news detection module performs event-level news detection on the event cluster to obtain an event cluster output.
[0013] To optimize the above technical solutions, the specific measures taken also include:
[0014] The above step 1 includes:
[0015] (1a) Feature extraction of news text: Use the pre-trained BERT model to perform entity recognition on the text in the news text, and then use Word2Vec to generate corresponding word vectors for each entity. After converting the text into vectors, the entities are connected according to semantic similarity to obtain a news subgraph represented by the vector;
[0016] (2b) Feature extraction of news images: Use the pre-trained scene graph generator SGG to extract features from the image data in the post news to obtain entities in the image, use Word2Vec to convert the entities into vectors, and then add the obtained vectors as nodes to the news subgraph in (1a);
[0017] (3c) Extraction of external knowledge features of news: For the news entities generated by BERT from news texts, ConceptNet is used to retrieve conceptual knowledge related to the entities to enhance the representation ability of the entities.
[0018] The above step 2 includes:
[0019] (2a) For the news subgraph represented by the vector, the semantic similarity between entity nodes is calculated;
[0020] (2b) After traversing all entity nodes, each entity is connected to its two corresponding entities with the highest similarity to obtain the corresponding multimodal subgraph.
[0021] Step 3 above includes:
[0022] (3a) Use BERT to identify key regions, celebrities, and events in news headlines and provide topic tags for the entire news post;
[0023] (3b) News articles with the same topic tag are sorted according to their timestamp information, and multiple news posts with close time are regarded as an event cluster.
[0024] The process of post-level news detection by the post-level news detection module in step 4 is as follows:
[0025] For each entity vector h = {h1,h2,...,h N}, The vector features are processed by the graph attention network GAT layer to obtain the mapping feature h = {h1,h2,...,h N}, Using the Matrix Perform node feature transformation, the formula is as follows:
[0026]
[0027] Among them, α ij represents the attention coefficient from node j to i, represents a shared attention mechanism, represents the LeakyReLU nonlinear activation function, is the set of neighbor nodes of node i;
[0028] The attention coefficient and its corresponding feature are weighted summed to obtain the output feature representation of each node:
[0029]
[0030] Where σ is the activation function, K is the number of attention heads, and W k is the weight matrix of the kth attention head;
[0031] Perform average pooling on the vectors of all nodes in the multimodal subgraph:
[0032]
[0033] in is the average value of all node vectors in the multimodal subgraph that pass through the GAT layer;
[0034] After performing the above processing on each multimodal subgraph, h of each multimodal graph is obtained. g ,h g After another layer of GAT, the post-level output is obtained after average pooling.
[0035] The event-level news detection module described in the above step 5 performs event-level news detection to obtain an event cluster output.
[0036] For event clusters The vector representation set Connect the vectors with edges in chronological order, and K nodes form the feature The relationship between the edges of the nodes is represented by the adjacency matrix X and A serve as the input of the GCN network layer and output a high-order feature matrix
[0037]
[0038] Where σ is a nonlinear activation function, A = A + I, I is the identity matrix, W is the training parameter matrix, and D is the degree matrix of A;
[0039] Node representation After normalization, the event cluster output is obtained
[0040] In the pre-training phase of the above-mentioned fake news detection model based on time-series aggregation multimodal graph neural network, stochastic gradient descent is applied to minimize the loss function of the model:
[0041]
[0042] When all training data are back-propagated, a training cycle ends.
[0043] The present invention has the following beneficial effects:
[0044] The fake news detection model of the present invention constructs a multimodal graph to perform double-layer detection of post-level and event-level news, which can achieve efficient modal information fusion, accurately capture the overall characteristics of post-level news, and at the same time consider aggregated news from a time perspective to form event-level news for causal logic error judgment, thereby improving the accuracy of fake news detection. The implementation steps are: news multimodal information feature extraction; multimodal information feature aggregation; event-level news construction; post-level news detection; event-level news detection. A fake news detection model training and testing scheme is given, and the detection effect of the model on fake news is tested. The present invention models multimodal information as a subgraph structure to solve the problem of fake news detection, and more comprehensively represents the semantic features of news; at the same time, the intrinsic semantic relationship and thematic relationship of news in the time dimension are modeled to achieve the recognition of event-level fake news; using the characteristics of the graph attention mechanism introduced by the graph neural network, the model can focus on the representation of different modal features to achieve a more accurate classification effect.
[0045] The present invention proposes a method of modeling multi-source news information as a subgraph structure, which comprehensively expresses news information from the aspects of semantics, external knowledge and vision, and makes up for the shortcomings of the traditional single-modal fake news detection model. This effectively solves the problem of incomplete feature representation in fake news detection and more accurately represents the news content.
[0046] The present invention constructs the intrinsic semantic relationship and thematic relationship of news in the time dimension and examines the logical errors of news at the event level. Compared with the existing multimodal fake news detection methods, this method realizes the identification of fake news at the event level.
[0047] This paper proposes a fake news detection method based on time-series aggregation multimodal graph neural network, which uses graph attention mechanism for targeted learning. The model can focus on the representation of different modal features to achieve more accurate classification results. Compared with the best baseline model, this method exceeds the most competitive benchmark KMGCN in terms of accuracy and recall. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 is a flow chart of the detection method of the present invention;
[0049] Figure 2 It is a network framework diagram of the present invention;
[0050] Figure 3 It is a comparison chart of classification performance using the present invention and other existing technologies;
[0051] Figure 4 This is the performance of the present invention under ablation experiment. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0053] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It is understood that the term "and / or" used in this article involves and covers any and all possible combinations of one or more of the associated listed items.
[0054] The present invention discloses a false news detection method based on time series aggregation multimodal graph neural network, which uses a pre-trained false news detection model based on time series aggregation multimodal graph neural network to detect false news. Figure 1-2 The process of fake news detection is described as follows:
[0055] Step 1: extract news multimodal information features, convert feature entities into vectors, and obtain news subgraphs represented by vectors;
[0056] First, the news text features are extracted. The pre-trained Bidirectional Encoder Representation from Transformers (BERT) model is used to perform entity recognition on the text in the news text. Then Word2Vec is used to generate the corresponding word vector for each entity. After converting the text into a vector, the entities are connected according to the semantic similarity to obtain the news subgraph represented by the vector.
[0057] Then, the news images are feature extracted. The pre-trained scene graph generator (SGG) is used to extract features from the image data in the post news to obtain the entities in the image. Word2Vec is used to convert the entities into vectors, and then the obtained vectors are added as nodes to the subgraph consisting of the news text.
[0058] Finally, the external knowledge features of news are extracted. Relevant concept-level knowledge is retrieved from ConceptNet. For news entities that have been generated from news texts using the pre-trained language model BERT, ConceptNet can be used to retrieve conceptual knowledge related to the entity and enhance the representation ability of the entity.
[0059] Step 2: Aggregate multimodal information features based on the news subgraph represented by the vector to obtain a multimodal subgraph of the news post.
[0060] (2a) Calculate the semantic similarity between entity nodes.
[0061] The nodes in step 1 are converted into multimodal subgraphs by constructing edges. In step 1, Word2Vec is used to match the entity nodes to obtain the vector representation corresponding to the entity, and then the semantic similarity between the two is calculated. Here, cosine similarity is used for calculation, and the calculation formula is expressed as:
[0062]
[0063] where e i and e j Represent different entity nodes respectively, ||e i || and ||e j || represents the modulus of two vectors, and · represents the dot product operation between vectors.
[0064] (2b) After traversing all entity nodes, each entity is connected to its two corresponding entities with the highest similarity to obtain the corresponding multimodal subgraph, i.e., the multimodal subgraph of the news post.
[0065] Step 3: Construct event-level news, i.e., event clusters, based on the multimodal subgraph of news posts.
[0066] (3a) Considering that news headlines often contain key information in the news, the present invention uses BERT to identify key regions, celebrities and events in news headlines, thereby providing topic tags for the entire news post.
[0067] (3b) Sort the news articles under each same topic tag according to the timestamp information, and regard multiple news posts with similar time as an event cluster.
[0068] First, we traverse all news topics and obtain a list of key entities. We analyze the entity list to get the number of news items under different key entities. Then, considering that news items under similar topics and close in time often have more similar information and are complementary in information, we sort news articles under each same topic label according to timestamp information. Finally, we regard K news posts with similar time as event clusters. p k Represents the post-level news at a certain point in time. In the present invention, K is set to 30.
[0069] Step 4: The post-level news detection module performs post-level news detection to obtain post-level output.
[0070] In step 2, each news post is processed through Word2Vec to obtain the vector h = {h1,h2,...,h N}, The vector features are processed through the graph attention network (GAT) layer to obtain the mapping feature h = {h1,h2,...,h N}, Using the Matrix Perform node feature transformation, the formula is as follows:
[0071]
[0072] Among them, α ij represents the attention coefficient from node j to i, represents a shared attention mechanism, represents the LeakyReLU nonlinear activation function, is the set of neighbor nodes of node i.
[0073] The normalized attention coefficient is weighted and summed with its corresponding feature. In order to stabilize the training process, a multi-head attention mechanism is used in the model, and the output feature of each node is expressed as:
[0074]
[0075] Where σ is the activation function, K is the number of attention heads, and W k is the weight matrix of the kth attention head.
[0076] Perform average pooling on the vectors of all nodes in the graph:
[0077]
[0078] in is the average of all node vectors in the multimodal subgraph that pass through the GAT layer.
[0079] After performing the above processing on each multimodal graph, h of each multimodal graph is obtained. g , repeat the above steps, h g After another layer of GAT, the post-level output is obtained after average pooling.
[0080] Step 5: The event-level news detection module performs event-level news detection and obtains event cluster output.
[0081] The event cluster has been obtained in step 3 Correspondingly, the vector representation set of event clusters is obtained through step 4 Connect these vectors with edges in chronological order, and K nodes form the feature The relationship between the edges of the nodes is represented by the adjacency matrix X and A serve as the input of the GCN network layer and output a high-order feature matrix As shown below:
[0082]
[0083] Where σ is a nonlinear activation function, A=A+I, I is the identity matrix, W is the training parameter matrix, and D is the degree matrix of A.
[0084] Node representation After normalization, the event cluster output is obtained
[0085]
[0086] The above fake news detection model based on time series aggregation multimodal graph neural network was pre-trained and tested as follows before use:
[0087] (1) Input the event cluster training set and train the fake news detection model.
[0088] In the training phase, stochastic gradient descent is applied to minimize the loss function of the model, and the double-layer news classification loss at the event level and post level is defined as:
[0089]
[0090] When all training data are back-propagated, a training cycle ends.
[0091] (2) Input the test set data into the network to test the model’s effectiveness in detecting fake news.
[0092] Input the test set to test the model effect. Accuracy, recall, F1score and precision are used to evaluate the model's effect on fake news detection. The calculation method is:
[0093]
[0094]
[0095]
[0096]
[0097] Among them, TP is the number of true news predicted as true, FN is the number of true news predicted as false, FP is the number of false news predicted as true, and TN is the number of false news predicted as false.
[0098] The effect of the present invention is further described below in conjunction with simulation experiments.
[0099] Simulation conditions and parameter settings:
[0100] The simulation experiment of the present invention is carried out on the simulation platform of Python3.7 and Pytorch1.9.0. The computer CPU model is AMD EPYC 7302, and the independent graphics card model is NVIDIAGeForce RTX 3090. The learning rate of network training is 0.001, and the Adam optimization algorithm is selected as the network training optimizer. In the design of the graph neural network, the input dimension of the first layer of GAT is 300, the output dimension is 16, and a multi-head attention mechanism is adopted, including 4 heads. The input dimension of the second layer of GAT is 64, and the output dimension is 2. The input dimension of the GCN layer is 64, and the output dimension is 2.
[0101] In an embodiment of the present invention, two public data sets are used to evaluate the results. Dataset 1 contains 20,015 news items, including 11,941 fake news items and 8,074 true news items. The data set contains various information, such as title, text, image, release time and website. The present invention utilizes title, text, image and time information. Dataset 2 is about event news in India from 2013 to 2021. The original data set contains 56,714 news items, including 18,914 fake news items and 37,800 true news items. The data set contains news text, image URL, news topic and date. The LDA (latent Dirichletallocation) algorithm is used to perform topic modeling on news sentences to achieve classification of news sentences. Since the image URL is not available, only news text, news topic and date information are used in the present invention.
[0102] Simulation content:
[0103] Figure 3 This is a comparison chart of fake news detection performance using the present invention and the existing baseline model under two data sets. Accuracy, recall, F1 and precision are used as performance indicators for comparison. Overall, in the two data sets, the performance of the method using multimodal information is often better than that of the method using single-modal information, which shows that the addition of multimodal information makes the model more advantageous, improves the robustness and generalization ability of the model, and verifies the necessity of designing a well-designed multimodal information model to capture the intrinsic correlation and semantic hierarchy across modalities.
[0104] In addition, among the four performance indicators of the two data sets, the present invention achieved the best performance, with an accuracy rate of more than 90%. In data set 1, the accuracy of the present invention exceeded the most competitive baseline method by 6%, and in data set 2, the accuracy of the present invention exceeded the most competitive baseline method by about 2.7%. This shows that the introduction of visual information and external knowledge can enhance the semantic information of news texts, and the introduction of timestamp information to construct event-level news on the same topic can enhance the recognition effect of fake news. The features of news subgraphs can be effectively aggregated through graph neural networks to enhance the performance of the model.
[0105] Figure 4is the performance of the present invention under ablation experiments, where w / o Time represents that the event-level detector constructed using timestamp information is not introduced into the model, w / o EK represents that external knowledge information is not added when constructing the news subgraph, w / o IMG represents that picture information in the news is not added when constructing the news subgraph, and w / o EK&IMG represents that image information and external knowledge information in the news are not added when constructing the news subgraph. The ablation separation of each modality of the present invention has caused the performance of the main body of the invention to decline to varying degrees, which confirms the effectiveness of each modality of the present invention. In comparison, the complete model that introduced multimodal information and time information achieved the best results, and was superior to several other ablation models in accuracy, recall and F1, which shows that the introduction of external knowledge and image information can improve the performance of the model. Secondly, introducing timestamp information to construct event-level news on the same topic can enhance the recognition of fake news. Compared with the other models, the model that removes time information has the largest performance drop, with a 6.23% difference in accuracy in Dataset 1. This shows the importance of introducing timestamp information. Multiple news event descriptions under the same topic at adjacent times can enhance each other's information and semantics, making the classification effect more significant. Removing image information or external knowledge also causes a certain degree of decline in the performance of the model, with a 2.26% and 1.49% difference in accuracy in Dataset 1, respectively. This shows that the addition of external knowledge or image information is necessary and is a factor that cannot be ignored when improving model accuracy.
[0106] Based on the above simulation results and ablation experimental analysis, the fake news detection method based on time-series aggregation multimodal graph neural network proposed in the present invention can achieve efficient modal information fusion, accurately capture the overall characteristics of post-level news, and at the same time consider aggregated news from a time perspective to form event-level news for causal logic error judgment, thereby improving the accuracy of fake news detection.
[0107] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
[0108] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. A false news detection method based on time series aggregation multimodal graph neural network, which uses a pre-trained false news detection model based on time series aggregation multimodal graph neural network to detect false news, characterized in that: The model performs fake news detection process including: Step 1: extract news multimodal information features, convert feature entities into vectors, and obtain news subgraphs represented by vectors; Step 2: Aggregate multimodal information features based on the news subgraph represented by the vector to obtain a multimodal subgraph of the news post; Step 3: Construct event-level news, i.e., event clusters, based on the multimodal subgraph of news posts; Step 4: The post-level news detection module performs post-level news detection on the multimodal subgraph of the news post to obtain a post-level output; Step 5: The event-level news detection module performs event-level news detection on the event cluster to obtain an event cluster output; The process of post-level news detection by the post-level news detection module in step 4 is as follows: For each entity vector h = {h1,h2,...,h N }, The vector features are processed through the graph attention network GAT layer to obtain the mapping features Using the Matrix Perform node feature transformation, the formula is as follows: Among them, α ij represents the attention coefficient from node j to i, represents a shared attention mechanism, represents the LeakyReLU nonlinear activation function, is the set of neighbor nodes of node i; The attention coefficient and its corresponding feature are weighted summed to obtain the output feature representation of each node: Where σ is the activation function, K is the number of attention heads, and W k is the weight matrix of the kth attention head; Perform average pooling on the vectors of all nodes in the multimodal subgraph: in is the average value of all node vectors in the multimodal subgraph that pass through the GAT layer; After performing the above processing on each multimodal subgraph, h of each multimodal graph is obtained. g ,h g After another layer of GAT, the post-level output is obtained after average pooling. The event-level news detection module in step 5 performs event-level news detection to obtain event cluster output, including: For event clusters The vector representation set Connect the vectors with edges in chronological order, and K nodes form the feature The relationship between the edges of the nodes is represented by the adjacency matrix X and A serve as the input of the GCN network layer and output a high-order feature matrix where σ is a nonlinear activation function, I is the identity matrix, W is the training parameter matrix, yes The degree matrix of Node representation After normalization, the event cluster output is obtained 2. According to claim 1, a fake news detection method based on time series aggregation multimodal graph neural network is characterized in that: The step 1 comprises: (1a) Feature extraction of news text: Use the pre-trained BERT model to perform entity recognition on the text in the news text, and then use Word2Vec to generate corresponding word vectors for each entity. After converting the text into vectors, the entities are connected according to semantic similarity to obtain a news subgraph represented by the vector; (2b) Feature extraction of news images: Use the pre-trained scene graph generator SGG to extract features from the image data in the post news to obtain entities in the image, use Word2Vec to convert the entities into vectors, and then add the obtained vectors as nodes to the news subgraph in (1a); (3c) Extraction of external knowledge features of news: For the news entities generated by BERT from news texts, ConceptNet is used to retrieve conceptual knowledge related to the entities to enhance the representation ability of the entities.
3. According to claim 1, a fake news detection method based on time series aggregation multimodal graph neural network is characterized in that: The step 2 comprises: (2a) For the news subgraph represented by the vector, the semantic similarity between entity nodes is calculated; (2b) After traversing all entity nodes, each entity is connected to its two corresponding entities with the highest similarity to obtain the corresponding multimodal subgraph.
4. According to claim 1, a fake news detection method based on time series aggregation multimodal graph neural network is characterized in that: The step 3 comprises: (3a) Use BERT to identify key regions, celebrities, and events in news headlines and provide topic tags for the entire news post; (3b) News articles with the same topic tag are sorted according to their timestamp information, and multiple news posts with close time are regarded as an event cluster.
5. According to claim 1, a fake news detection method based on time series aggregation multimodal graph neural network is characterized in that: In the pre-training phase, the fake news detection model based on time-series aggregation multimodal graph neural network applies stochastic gradient descent to minimize the loss function of the model: When all training data are back-propagated, a training cycle ends.
Citation Information
Patent Citations
A Fake News Detection Method Based on Generative Adversarial Networks and Multimodal Fusion
CN113221872B
