A false news detection device based on graph transformer and information bottleneck

Abstract

The application belongs to the field of artificial intelligence and natural language processing, and relates to a false news detection method based on graph Transformer and information bottleneck. First, a new graph position encoding calculation formula is designed to construct a propagation relationship graph based on position encoding. Second, the information bottleneck principle is used to learn a denoising attention graph. Finally, the constructed propagation relationship graph and denoising attention graph are used as a mask graph to mask the self-attention layer of the Transformer. Through the masking mechanism, on the one hand, the negative effects of noise information can be effectively eliminated, and on the other hand, the propagation structure and long sequence dependency of news can be effectively described. The method is applied to a real news propagation network to identify false news in news data with high accuracy.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence and natural language processing, and relates to a fake news detection device based on graph Transformer and information bottleneck. Background Technology

[0002] In the digital age, we live in an era of information overload. The internet, social media, and digital platforms generate and disseminate vast amounts of information daily. While this rapid flow of information provides convenience, it also brings problems such as fake news, misleading information, and counterfeit information. This deceptive information not only misleads individuals but can also have profound negative impacts on society, politics, and the economy. Therefore, inventing an effective fake news detection device plays a crucial role in preventing the spread of fake news by various social institutions, maintaining social harmony, and upholding public trust in these institutions.

[0003] Traditional news detection methods primarily rely on manual review and simple keyword matching, but these methods fall short when handling large-scale data. With technological advancements, machine learning and deep learning have achieved significant success in many fields, particularly in image recognition, speech recognition, and natural language processing. Based on these advanced technologies, various machine learning and deep learning-based schemes and devices have been proposed for detecting fake news.

[0004] Among these methods, Graph Neural Networks (GNNs) have received widespread attention. GNNs can operate directly on graphs, thereby capturing the topological structure of data and the relationships between nodes. Based on this, existing schemes usually regard the original news as the root node. Users participating in the discussion of this news are regarded as other nodes, and the comment relationships between users are regarded as the edges between nodes. However, these methods still have the following drawbacks: (1) Since existing GNNs can usually only capture the local graph topological structure, they cannot model the long-sequence dependency features in the process of fake news propagation. The long-sequence dependency features of propagation usually characterize important features such as the evolution and formation of comment topics. The lack of characterization of the long-sequence dependency features makes it difficult to accurately discover the propagation pattern of fake news, thereby reducing the success rate of fake news detection. (2) In the real world, there are a large number of malicious users or social bots in the news. They usually publish some malicious comments to manipulate the spread of fake news, thereby introducing a lot of noise information. This noise information can easily deceive the detection model, causing the model to make incorrect predictions. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the existing technology, the purpose of this application is to provide a fake news detection device based on graph Transformer and information bottleneck.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A fake news detection device based on graph Transformer and information bottleneck includes the following steps:

[0008] S1: Construct a news dissemination graph G = {V, E, X} for each news item, where V = {v0, v1, ..., v n Let {} represent the set of n nodes, E represent the set of edges, and X = {x0, x1, ..., xn}. n} represents the initial node features of the news dissemination graph;

[0009] S2: Construct a propagation relationship graph G based on position encoding p The specific construction method is shown in formula (1):

[0010]

[0011] Where P ij Let matrix A P In the i-th row and j-th column of matrix A P To propagate the relationship graph G p The adjacency matrix, d ij Represents node v i With node v j The shortest path value between, d i Represents node v i The shortest path value to the root node v0, d j Represents node v j The shortest path value to the root node v0;

[0012] S3: Construct a denoised attention map G s ,include:

[0013] 31) Input graph G into an existing standard Transformer to obtain a low-dimensional vector representation Z of all nodes, Z = {z0, z1, z2, ..., z}. n} represents the low-dimensional vector representation of a node;

[0014] 32) Construct a Bernoulli sampling probability π between each pair of nodes. ij As shown in formula (3):

[0015] π ij =Sigmoid(MLP(z i ,z j (3)

[0016] Using Bernoulli sampling, calculate any pair of nodes v i With node v jAre there attentional connections between them, as shown in Equation (4)?

[0017]

[0018] Among them, S ij Let matrix A s The matrix A is in the i-th row and j-th column. s For fine-grained denoising attention map G s The adjacency matrix of G is given, where t is the temperature parameter with a value in (0,1), ∈ ~Bernoulli(0,1), and is a value randomly sampled from the Bernoulli distribution; at the same time, the node features X in G are updated with the Z learned at this time;

[0019] S4: Construct the self-attention module of the masking graph Transformer, as shown in formula (5):

[0020]

[0021] in, Let be a learnable matrix, where d represents the dimension of the low-dimensional representation matrix of the output, and h represents the h-th attention head in the multi-head attention mechanism. H Let A represent the dimension of each attention head, T represent the matrix transpose operation, and A represent the dimension of each attention head. i ∈{A s A p A f}, where A s A p , and A f =1 |V| × |V| Representing graph G s The adjacency matrix of graph G p Adjacency matrix and a fully connected graph G f softmax is a non-linear activation function, and Mask() is a masking function, defined as shown in formula (6):

[0022] Mask(x,λ)=x+ξλ(6)

[0023] Where ξ is a sufficiently large value, formula (6) means directly applying the noise reduction attention G s and the propagation relationship graph G p The information is embedded into the self-attention matrix;

[0024] S5: Construct the feedforward layer module of the masking graph Transformer to generate the node representation Z = {z0, z1, ..., z} n}, specifically as shown in formulas (7) and (8):

[0025]

[0026] Z=FFN(X′)=ReLU(X′W1)W2(8)

[0027] in, r is the dimension of the multilayer perceptron output. as well as Let H be the learnable parameter matrix, H represent the total number of attention heads, ReLU represent the non-linear activation function, and Z represent the non-linear activation function. final Represented as nodes;

[0028] S6: The graph-level representation is constructed as shown in formula (9):

[0029]

[0030] S6: The graph-level representation is constructed as shown in formula (9):

[0031]

[0032] Where Z represents a node;

[0033] S7: Loss function construction and optimization, including:

[0034] 71) Constructing KL divergence loss to control G s The sparsity of is shown in Equation (10).

[0035]

[0036] Among them, E full Let G represent a fully connected graph constructed from a set of nodes V. f The edge table, ρ is the control chart G s The parameter indicating the sparsity of the region;

[0037] Construct the predictor loss function L cls As shown in formulas (11) and (12):

[0038]

[0039] Where MLP stands for Multilayer Perceptron, and Softmax is a non-linear activation function. The predicted labels;

[0040]

[0041] Y∈{0,1} represents the predicted label;

[0042] 72) The final optimized objective function is shown in formula (13):

[0043] Lloss =L KL +L cls (13)

[0044] S8: Predicted output on the test set.

[0045] Furthermore, in step S1, the news itself is considered a root node v0, and all users who participate in commenting are considered other nodes in the news propagation graph. If user v i If a user directly comments on the news item v0, then that user node v... i Construct an edge connecting user v to the news root node v0; if user v j The comment replied to user v i The comment is for node v. i and v j Construct an edge connecting the nodes; for each node v i initial features x i By inputting the news or comment text corresponding to the node into the BERT model for learning, a low-dimensional vector representation of each node can be obtained, i.e., x. i .

[0046] In a preferred embodiment of this application, ξ is set to 1000.

[0047] In a preferred embodiment of this application, ρ is set to 0.8.

[0048] In a preferred embodiment of this application, the training process in S7 is performed on the training set. 75% of the news dissemination images in the collected data are used as the training set, and the remainder is used as the test set. Training stops when the following two conditions are met: 1) when the number of training iterations exceeds the preset value t = 1000, and 2) when the loss function decreases below q after 20 consecutive iterations. The setting of q varies in different data and should be adjusted according to actual needs. After training stops, the model parameters are saved for prediction on the test set.

[0049] Further, in step S8, the predicted label values ​​are output. In this context, 0 represents that the news is true, and 1 represents that the news is fake.

[0050] Compared with the prior art, this application has the following beneficial effects:

[0051] (1) Based on the information bottleneck principle, this application designs a unique noise reduction mechanism for graph Transformers. By introducing the information bottleneck into the self-attention mechanism of graph Transformers, it effectively removes task-irrelevant attention connections in the self-attention module, thereby producing interpretable and robust results and effectively reducing noise interference in the collected data. Therefore, the method of this application is particularly suitable for deployment in social platforms with noisy environments.

[0052] (2) This application proposes a new positional encoding method based on propagation structure and proposes a propagation relationship graph based on positional encoding. By using this graph as a mask graph in the Transformer self-attention mechanism, the Transformer can capture the structural features of propagation while capturing long sequence dependency features, enabling the model to capture the propagation structure and long sequence dependency, and further improving the accuracy and reliability of fake news detection.

[0053] (3) The method of this application was applied to the real news dissemination network, which enabled the identification of fake news in news data with a high accuracy rate, providing theoretical guidance and technical support for the formulation of effective fake news prevention and control measures. Attached Figure Description

[0054] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0055] Figure 1 This is a flowchart of the method in Embodiment 1 of this application;

[0056] Figure 2 This is a detailed illustration of the method in Embodiment 1 of this application. Detailed Implementation

[0057] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0058] Example 1

[0059] like Figure 1 As shown, the fake news detection method based on graph Transformer and information bottleneck includes the following steps:

[0060] S1: Construct a news dissemination graph G = {V, E, X} for each news item. Where V = {v0, v1, ..., v...} nLet} represent the set of n nodes. Let E represent the set of edges. This scheme treats the news item itself as a root node v0, and all users who participate in commenting as other nodes in the graph. If user v... i If a user directly comments on the news item v0, then that user node v... i Construct an edge connecting user v to the news root node v0. j The comment replied to user v i The comment is for node v. i and v j Construct an edge connecting the x and xy elements. X = {x0, x1, ..., xy} n} represents the initial node features of the news dissemination graph. For each node v i initial features x i This solution inputs the news or comment text corresponding to the node into an existing BERT model for learning, thereby obtaining a low-dimensional vector representation of each node, i.e., x. i .

[0061] S2: Construct a propagation relationship graph G based on position encoding p This step aims to take the G constructed in step S1 as input and output a fine-grained propagation graph G. p The specific construction method is shown in formula (1):

[0062]

[0063] Where P ij Let matrix A P The matrix A is located in the i-th row and j-th column. P To propagate the relationship graph G p The adjacency matrix of d. ij Represents node v i With node v j The shortest path value between them. d i Represents node v i The shortest path value to the root node v0. d j Similarly.

[0064] S3: Construct a denoised attention map G s This step is an iterative update step, i.e., graph G. s The learned graph G is obtained through iterative updates. First, the graph G is input into the existing standard Transformer, as shown in Equation (2):

[0065] Z = Transformer(G) (2)

[0067] Where Z = {z1, z2, ..., z} nLet} be a low-dimensional vector representation of the nodes. Next, a Bernoulli sampling probability π is constructed between each pair of nodes. ij As shown in formula (3):

[0068] π ij =Sigmoid(MLP(z i ,z j (3)

[0069] Next, using Bernoulli sampling, the value of any pair of nodes v is calculated. i With node v j Are there attentional connections between them, as shown in Equation (4)?

[0070]

[0071] Among them, S ij Let matrix A s The i-th row and j-th column of matrix A. s For fine-grained denoising attention map G s The adjacency matrix of G is given. t is a temperature parameter taking values ​​in (0,1), ∈ ~Bernoulli(0,1), and is a value randomly sampled from a Bernoulli distribution. Simultaneously, the node features X in G are updated with the learned Z.

[0072] S4: Construct the self-attention module of the masking graph Transformer. This step aims to build upon the G obtained in step S2. p And Gs obtained in step S3, construct the self-attention module of the masking graph Transformer. Specifically, as shown in equation (5):

[0073]

[0074] in, Let be a learnable matrix, where d represents the dimension of the low-dimensional representation matrix of the output, and h represents the h-th attention head in the multi-head attention mechanism. H Let A represent the dimension of each attention head, T represent the matrix transpose operation, and A represent the dimension of each attention head. i ∈{A s A p A f}, where A s A p , and A f =1 |V| × |V| Representing graph G s The adjacency matrix of graph G p Adjacency matrix and a fully connected graph G fsoftmax is a non-linear activation function, and Mask() is a masking function, defined as shown in formula (6):

[0075] Mask(x,λ)=x+ξλ (6)

[0076] Where ξ is a sufficiently large value, set to 1000 in this embodiment, and formula (6) means directly applying the noise reduction attention G s and the propagation relationship graph G p The information is embedded into the self-attention matrix;

[0077] S5: Construct the feedforward layer module (FFN) of the masking graph Transformer. This step aims to generate the node representation Z = {z0, z1, ..., z...} based on the self-attention module constructed in step S4. n}, specifically as shown in formulas (7) and (8):

[0078]

[0079] Z=FFN(X′)=ReLU(X′W1)W2 (8)

[0080] in, r is the dimension of the multilayer perceptron output. as well as Here, H represents the total number of attention heads, ReLU represents the non-linear activation function, and Z represents the node representation.

[0081] S6: Graph-level representation construction. This step aims to generate the graph-level representation s based on the node representation Z obtained in S5. Specifically, this step uses a ReadOut function to obtain s, as shown in formula (9):

[0082]

[0083] S7: Loss Function Construction and Optimization. This step aims to build upon the π from step S3. ij And in S6, s, design the final objective function and optimization regulation. First, construct the KL divergence loss to control G. s The sparsity of is shown in Equation (10).

[0084]

[0085] Among them, E full Let G represent a fully connected graph constructed from a set of nodes V. f The edge table. ρ is the control chart G. s The parameter for sparsity is set to ρ = 0.8 in this embodiment.

[0086] Furthermore, this embodiment also constructs the predictor loss function L. cls As shown in formulas (11) and (12):

[0087]

[0088] MLP stands for Multilayer Perceptron. Softmax is a non-linear activation function. The predicted labels are 0 for true news and 1 for fake news.

[0089]

[0090] Y∈{0,1} represents the predicted label, where 0 indicates the news is true and 1 indicates it is false. Based on this, the final optimized objective function is shown in formula (13):

[0091] L loss =L KL +L cls (13)

[0092] The above training process is all performed on the training set. In this scheme, 75% of the news dissemination images collected are used as the training set, and the remainder is used as the test set. Training stops when the following two conditions are met: (1) When the number of training iterations exceeds the preset value t = 1000. (2) When the loss function decreases below q after 20 consecutive iterations. The setting of q varies in different data and should be adjusted according to actual needs. After training stops, the model parameters are saved for prediction on the test set.

[0093] S8: Predict the output on the test set. This step takes the test set as input and outputs the predicted label values. In this system, 0 represents that the news is true, and 1 represents that the news is fake. This completes the task of detecting false information.

[0094] Example 2

[0095] Table 1 shows the size of the real-world network dataset used for experimental validation:

[0096] Table 1 shows the real news dissemination graph dataset used for experimental verification.

[0097] Politifact 314 157 41054 40740 Gossipcop 5464 2732 314262 308798

[0098] Figure 2 The diagram shows a detailed illustration of the method in this embodiment. Specifically, the present invention first constructs G based on the propagation structure. p And use the mutual information bottleneck criterion to learn the denoised attention graph G. s Next, the constructed G p and Gs As a masking map for the Transformer's self-attention module, this allows the Transformer to effectively capture propagation structure features and reduce noise interference. Finally, by minimizing the KL loss L... KL And predicted loss L cls The model can be continuously optimized.

[0099] Table 2 shows the experimental results on Politifact. Accuracy (ACC), Precision (Prec.), Recall (Rec.), and F1 Score (F1) are the four evaluation metrics. The method proposed in this embodiment is named PSGT. Experimental results show that, compared with existing methods, the proposed solution achieves the highest accuracy in detecting false information.

[0100] Table 2 shows the experimental results on Politifact.

[0101] CSI 0.734 0.672 0.55 0.688 Bert-MLP 0.853 0.892 0.833 0.825 SpaCy-MLP 0.375 0.124 0.136 0.272 GCN 0.833 0.958 0.766 0.831 GAT 0.895 0.93 0.9 0.89 GraphGPS 0.903 0.9 0.818 0.857 BiGCN 0.854 0.96 0.8 0.851 UPFD 0.875 0.9 0.9 0.866 GACL 0.875 0.961 0.833 0.871 EBGCN 0.896 0.898 0.909 0.891 GSAT 0.854 0.947 0.75 0.852 PSGT 0.917 0.966 0.918 0.922

[0102] Table 3 shows the experimental results on Gossipcop. Accuracy (ACC), Precision (Prec.), Recall (Rec.), and F1 Score (F1) are the four evaluation metrics. The proposed method is named PSGT. Experimental results show that, compared with existing methods, the proposed method achieves the highest accuracy in false information detection.

[0103] Table 3 shows the experimental results on Gossipcop.

[0104] CSI 0.866 0.892 0.84 0.866 Bert-MLP 0.962 0.968 0.954 0.961 SpaCy-MLP 0.501 0.498 0.534 0.333 GCN 0.957 0.931 0.984 0.957 GAT 0.961 0.945 0.971 0.96 GraphGPS 0.968 0.972 0.952 0.96 BiGCN 0.951 0.923 0.982 0.953 UPFD 0.965 0.972 0.957 0.965 GACL 0.976 0.971 0.972 0.976 EBGCN 0.964 0.966 0.962 0.963 GSAT 0.956 0.949 0.962 0.955 PSGT 0.98 0.974 0.987 0.98

[0105] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A method for detecting fake news based on graph Transformer and information bottleneck, characterized in that, Includes the following steps: S1: Construct a news dissemination graph G={V, E, X} for each news item, where V={v0, v1, …, v n Let} represent the set of n nodes, E represent the set of edges, and X = {x0, x1, …, x} n } represents the initial node features of the news dissemination graph; S2: Construct a propagation relationship graph G based on position encoding. p The specific construction method is shown in formula (1): P ij = (1) Where P ij Let matrix A P In the i-th row and j-th column of matrix A P To propagate the relationship graph G p The adjacency matrix, d ij Represents node v i With node v j The shortest path value between, d i Represents node v i The shortest path value to the root node v0, d j Represents node v j The shortest path value to the root node v0; S3: Construct a denoised attention map G s ,include: 31) Input graph G into an existing standard Transformer to obtain a low-dimensional vector representation Z of all nodes, Z={z0,z1, …, z n } represents the low-dimensional vector representation of a node; 32) Construct a Bernoulli sampling probability π between each pair of nodes. ij As shown in formula (3): π ij = Sigmoid (MLP (of i , With j )) (3) Using Bernoulli sampling, calculate any pair of nodes v i With node v j Are there attentional connections between them, as shown in Equation (4)? S ij = Bernoulli (π ij )≈sigmoid ( (log )) (4) Among them, S ij Let matrix A s The matrix A is in the i-th row and j-th column. s For fine-grained denoising attention map G s The adjacency matrix, where t is the temperature parameter taking values ​​in (0,1). ~Bernoulli (0, 1) represents a value randomly sampled from the Bernoulli distribution; simultaneously, the node features X in G are updated with the Z learned at this time. S4: Construct the self-attention module of the masking graph Transformer, as shown in formula (5): (X) = softmax (Mask ( , A i )) (5) in, =X , =X , ∈ , ∈ , ∈ Let be a learnable matrix, where d represents the dimension of the low-dimensional representation matrix of the output, and h represents the h-th attention head in the multi-head attention mechanism. H Let A represent the dimension of each attention head, T represent the matrix transpose operation, and A represent the dimension of each attention head. i ∈{A s A p A f }, where A s A p , and A f =1 |V| |V| Representing graph G s The adjacency matrix of graph G p Adjacency matrix and a fully connected graph G f softmax is a non-linear activation function, and Mask() is a masking function, defined as shown in formula (6): Mask (x, λ) = x + ξλ (6) Where ξ is a sufficiently large value, formula (6) means directly applying the noise reduction attention G s and the propagation relationship graph G p The information is embedded into the self-attention matrix; S5: Construct the feedforward layer module of the masking graph Transformer to generate the node representation Z=={z0, z1, …, z n }, specifically as shown in formulas (7) and (8): （7） Z=FFN( )=ReLU( W1)W2 (8) in, =X , ∈ , W1∈ , where r is the dimension of the multilayer perceptron output. ∈ and W2∈ Here, H represents the total number of attention heads, ReLU represents the non-linear activation function, and Z represents the node representation; S6: The graph-level representation is constructed as shown in formula (9): （9） S7: Loss function construction and optimization, including: 71) Constructing KL divergence loss to control G s The sparsity of is shown in formula (10): L KL = （10） Among them, E full Let G represent a fully connected graph constructed from a set of nodes V. f edge table, For control chart G s The parameter indicating the sparsity of the region; Construct the predictor loss function L cls As shown in formulas (11) and (12): =softmax(MLP(s)) (11) Where MLP stands for Multilayer Perceptron, and Softmax is a non-linear activation function. ∈{0, 1} represents the predicted label; L cls (AND, ) = -(Ylog +(1-Y)log(1- )) (12) Y∈{0, 1} represents the predicted label; 72) The final optimized objective function is shown in formula (13): L loss = L KL + L cls (13) S8: Predict the output on the test set; In step S1, the news itself is considered a root node v0, and all users who participate in the comments are considered other nodes in the news propagation graph. If user v i If a user directly comments on the news item v0, then that user node v... i Construct an edge connecting user v to the news root node v0; if user v j The comment replied to user v i The comment is for node v. i and v j Construct an edge connecting the nodes; for each node v i initial features x i The news or comment text corresponding to the node is input into the BERT model for learning, resulting in a low-dimensional vector representation of each node, i.e., x. i .

2. The method for detecting fake news based on graph Transformer and information bottleneck as described in claim 1, characterized in that, ξ is set to 1000.

3. The method for detecting fake news based on graph Transformer and information bottleneck as described in claim 1, characterized in that, Set as .

4. The method for detecting fake news based on graph Transformer and information bottleneck as described in claim 1, characterized in that, In S7, the training process is conducted entirely on the training set. 75% of the collected news dissemination images are used as the training set, and the remainder is used as the test set. Training stops when the following two conditions are met: 1) the number of training iterations exceeds the preset value t=1000; 2) the loss function decreases below q after 20 consecutive iterations. The setting of q varies in different datasets and should be adjusted according to actual needs. After training stops, the model parameters are saved for prediction on the test set.

5. The method for detecting fake news based on graph Transformer and information bottleneck as described in claim 1, characterized in that, In step S8, the predicted label values ​​are output. ∈{0, 1}, where 0 represents that the news is true and 1 represents that the news is fake.

Images