Multi-modal false news detection method based on content and external facts

Abstract

Aiming at the problem of how to improve the accuracy of a false news detection model, the invention designs a multi-modal false news detection method based on content and external facts. In order to enable the two modes to better focus on own preference parts and eliminate other redundant information as far as possible, a deep heterogeneous graph convolutional network (HDGCN) is introduced as a preference learning device, which can generate own required preference perception for the two models and reduce redundant information. Meanwhile, in order to more effectively model the node representation of the graph neural network, style words in the news are expanded by semantic mining, entity words in the news are expanded by entity links, and the obtained five word representations are jointly used as the node representation of the graph network. Finally, experiments on a public data set show the effectiveness of the model. In the field of false news detection, an idea and a wide application prospect are provided.

Description

Technical Field

[0001] The present invention designs a multimodal false news detection method based on content and external facts, which relates to the technical fields of deep learning and natural language processing. Background Art

[0002] Fake news detection technology is a popular direction in the field of natural language processing. Fake news detection refers to judging the truth or falsity of a given news article by combining the news content of the article with social contextual content and external knowledge.

[0003] In recent years, with the continuous development of social media, the spread of fake news has become more rapid and widespread. Fake news always makes some modifications to the correct statements, which is very deceptive and indistinguishable. Fake news can bring many harms, such as its widespread spread in different fields such as politics and public health, posing a huge threat to network security and human society. Therefore, there is an increasing demand for research on fake news detection.

[0004] Generally speaking, the research methods of fake news detection are mainly divided into the following categories: news content-based methods, external fact-based methods, and social context-based methods. Content-based methods usually regard fake news detection as a feature recognition task, and use language models to verify the authenticity of news only based on text patterns (such as writing style). However, content-based methods usually have poor generalization and interpretability. External fact-based methods usually model the task as a reasoning process, provide external evidence to investigate the authenticity of the news, and the model needs to discover and integrate useful information in the given evidence for fact verification. The key difference between these two methods is that they have different preferences for text clues. For example, the former may pay more attention to style words (such as anger, shock, etc.) and symbols (such as exclamation marks and question marks) in news texts, while the latter may pay more attention to words that represent facts (such as exact time, names, place names, etc.). Therefore, this paper considers integrating the two methods to detect news, and one of the keys is how to let the model learn the preferred part that is beneficial to itself and eliminate other redundant information as much as possible. Summary of the invention

[0005] The present invention provides a multimodal fake news detection method based on content and external facts to solve the above problems, and its main purpose is to improve the accuracy of the fake news detection model. The key point is how to make the two methods better focus on their own preference parts. To this end, we introduced a deep heterogeneous graph convolutional network (HDGCN) as a preference learner, which can generate the required preference perception for the two models and reduce redundant information. At the same time, in order to more effectively model the node representation of the graph neural network, we used semantic mining to expand the style words in the news, and used entity links to expand the entity words in the news. The five word representations obtained are used together as the node representation of the graph network.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] Step 1: Extract style words W from news texts through Bert s , extract entity words W from news texts through LAC and TextSmart e , extract style words W through HowNet and WordNet s The corresponding semantic word W sk , get entity word W through entity linking e The corresponding entity word W ek ;

[0008] Step 2: Introduce a deep heterogeneous graph convolutional network as a preference learning module, input the five words extracted from the news text into the preference learning module, and obtain the content preference perception m of the word feature vector respectively. P and fact preference perception m F ;

[0009] Step 3: Use the attention mechanism to perceive the obtained content preference m P It is fused with the content-based model, and the final fusion feature vector is represented as p;

[0010] Step 4: Use the attention mechanism to perceive the fact preference m F It is fused with the model based on external facts, and then sent to the reasoning module together with the returned facts to obtain the final fused feature vector represented as f;

[0011] Step 5: Concatenate the output vectors of the obtained content-based and external fact-based models and input them into the multi-layer perceptron MLP to obtain the final prediction

[0012] Step 6: Use three loss functions to calculate the final loss L, namely the cross entropy loss function Cosine similarity L between content preference and fact preference cos, the loss obtained after exchanging the input preferences of the two models

[0013] It should be noted that:

[0014] The semantic word W in step 1 sk By mining style words W s The semantic information in W is obtained, including hyponyms, synonyms, and semantics. ek By extracting entity words W from external knowledge bases CN_DBpedia and WiKi knowledge base e Obtained by matching with the entity words existing in the knowledge base;

[0015] The preference learning module in step 2 is mainly composed of a deep heterogeneous graph convolutional network. The training steps of this module are as follows:

[0016] (1) Initialize the edge weight A between adjacent nodes by calculating the cosine similarity of entity pairs (0) (i,j)∈(0,1):

[0017]

[0018] in, and is the initialized node feature, A (0) (i,j)∈(0,1) is the edge weight connecting the i-th and j-th nodes;

[0019] (2) Adopt a hierarchical attention strategy to process different types of nodes separately and then aggregate the interactive features:

[0020]

[0021] Where, T = {W s ,W sk ,W e ,W ek ,W o}; yes The sub-matrix of represents a symmetric normalized matrix, and A is the weighting matrix; The rows represent the nodes in the graph, and the columns represent the types of nodes; represents the attention weight of type τ in the lth layer; σ is the ReLU activation function; is the learnable transfer matrix; is the initialization vector of the τ-type node;

[0022] (3) Connect all layers with different attention scores in the graph convolutional network. Each layer in the convolutional network accepts the output of all previous layers as input; that is, H l+1 It consists of graph convolutional transformations from all previous layers:

[0023] H (l+1) =T(α l+1 *f(H (l) ,W (l) ),α l *H (l) ,…,α0*H (0) ) (3)

[0024] Where T is a connection function, which can be an addition function or a concatenation function; α i is the attention score of the layer; f is the convolution function;

[0025] (4) Obtain the final preference score by normalizing the relevance of each word feature vector:

[0026]

[0027]

[0028] Where m P 、m F is the final preference score, are the preference scores of the i-th layer and the j-th layer respectively.

[0029] The main content of this invention is to propose a multimodal fake news detection method based on content and external facts. We selected three general models based on content and three general models based on external facts as basic models for comparative experiments, and also verified the effectiveness of the proposed network on public datasets Weibo and Twitter. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a network structure diagram of the present invention.

[0031] Figure 2 This is the structural diagram of the deep heterogeneous graph convolutional network of the present invention.

[0032] Figure 3 This is a network structure diagram of the present invention integrating content preference perception.

[0033] Figure 4 It is a network structure diagram of the present invention integrating external fact preference perception. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings:

[0035] Figure 1 The network structure diagram of the multimodal fake news detection method based on content and external facts. Through semantic mining, entity linking and other methods, five word representations in the news text are obtained, namely, style words W s and its corresponding semantic knowledge word W sk , entity word W e and its corresponding entity link word W ek And other words W o ; Feature extraction and initialization are performed through the pre-trained language model BERT to obtain the corresponding vector representation H 0 ∈R n×d (d represents the dimension of each word feature vector), H 0 =[H s ,H e ,H ek ,H sk ,H o ]; then use the deep heterogeneous graph convolutional network to learn the preference and generate content preference perception m P and external fact preference perception m F ; Fuse it with the content-based model and the external fact-based model to obtain the fused feature vectors p and f; finally, send p and f to the multi-layer perceptron MLP to obtain the final prediction

[0036] Figure 2 This is the structural diagram of the deep heterogeneous graph convolutional network of the present invention. Compared with the traditional graph convolutional network (GCN) and heterogeneous graph convolutional network (HGCN), HDGCN has significantly improved the effect of extracting and fusing different features. Specifically, on the one hand, in order to obtain more useful information in heterogeneous graphs, HDGCN introduces a hierarchical attention mechanism, which can capture the different contributions of different relationships, different node types and different layers to node embedding; on the other hand, with the sharp increase in the number of nodes and corresponding relationships in the graph, it will lead to the problems of "neighborhood explosion" and over-smoothing. To this end, HDGCN introduces an adjacent domain sampling strategy to control the number of adjacent aggregated nodes. At the same time, the sampling is repeated in each layer, which not only retains the node information in the graph, but also alleviates the problems of "neighborhood explosion" and over-smoothing.

[0037] Figure 3 This is the network structure diagram of the present invention integrating content preference perception. Content-based models usually use text feature extractors to obtain vectors for final prediction, such as LSTM, BERT, etc. In order to better integrate preference perception with the corresponding model, we use the attention mechanism. For example, after the news text is extracted by BERT and other features, the vector [p1, p2, ..., p n], then the final fusion feature vector can be expressed as:

[0038]

[0039] in is the preference score of the i-th layer; p i is the vector feature of the i-th layer;

[0040] Figure 4 is the network structure diagram of the present invention integrating external fact preference perception. Models based on external facts usually retrieve from external fact sources to collect relevant articles (or evidence). We also use the attention mechanism to integrate the external fact-based preference perception with the corresponding model. Let the returned fact articles be represented as [d1, d2, …, d n ], we first use the attention mechanism to fuse the preference perception with the news text, and then feed it together with the returned facts into the reasoning module, which is usually designed to capture the complex interactions between text and factual evidence, such as consistency and conflict. The output vector of the reasoning module is used as the final fusion vector:

[0041]

[0042] in is the preference score of the i-th layer; q i is the vector feature of the i-th layer;

[0043] Experimental Results

[0044] Table 1: Single model comparison

[0045]

[0046] Table 2: Fusion model comparison

[0047]

Claims

1. A multimodal fake news detection method based on content and external facts, characterized by The following steps are involved: Step 1: Extract style words W from news texts through Bert s , extract entity words W from news texts through LAC and TextSmart e , extract style words W through HowNet and WordNet s The corresponding semantic word W sk , get entity word W through entity linking e The corresponding entity word W ek ; Step 2: Introduce a deep heterogeneous graph convolutional network as a preference learning module, input the five words extracted from the news text into the preference learning module, and obtain the content preference perception m of the word feature vector respectively. P and fact preference perception m F ; Step 3: Use the attention mechanism to perceive the obtained content preference m P It is fused with the content-based model, and the final fusion feature vector is represented as p; Step 4: Use the attention mechanism to perceive the fact preference m F It is fused with the model based on external facts, and then sent to the reasoning module together with the returned facts to obtain the final fused feature vector represented as f; Step 5: Concatenate the output vectors of the obtained content-based and external fact-based models and input them into the multi-layer perceptron MLP to obtain the final prediction Step 6: Use three loss functions to calculate the final loss L, namely the cross entropy loss function Cosine similarity L between content preference and fact preference cos , the loss obtained after exchanging the input preferences of the two models 2. The multimodal false news detection method based on content and external facts according to claim 1 is characterized by: The semantic word W in step 1 sk By mining style words W s The semantic information in W is obtained, including hyponyms, synonyms, and semantics. ek By extracting entity words W from external knowledge bases CN_DBpedia and WiKi knowledge base e Obtained by matching with the entity words existing in the knowledge base; The preference learning module in step 2 is mainly composed of a deep heterogeneous graph convolutional network. The training steps of this module are as follows: (1) Initialize the edge weight A between adjacent nodes by calculating the cosine similarity of entity pairs (0) (i, j)∈(0, 1): in, and is the initialized node feature, A (0) (i, j)∈(0, 1) is the edge weight connecting the i-th and j-th nodes; (2) Adopt a hierarchical attention strategy to process different types of nodes separately and then aggregate the interactive features: Where, T = {W s , W sk , W e , W ek , W o }; yes The sub-matrix of represents a symmetric normalized matrix, and A is the weighting matrix; The rows represent the nodes in the graph, and the columns represent the types of nodes; represents the τ-type attention weight in the first layer; σ is the ReLU activation function; is the learnable transfer matrix; X τ is the initialization vector of the τ-type node; (3) Connect all layers with different attention scores in the graph convolutional network. Each layer in the convolutional network accepts the output of all previous layers as input; that is, H l+1 It consists of graph convolutional transformations from all previous layers: H (l+1) =T(α l+1 *f(H (l) ,W (l) ),α l *H (l) ,...,α0*H (0) ) (3) Where T is a connection function, which can be an addition function or a concatenation function; α i is the attention score of the layer; f is the convolution function; H (l) is the interaction feature of the first layer; (4) Obtain the final preference score by normalizing the relevance of each word feature vector: Where m P 、m F is the final preference score, are the preference scores of the i-th layer and the j-th layer respectively.

Images