A correlation-based fake news detection method with adaptive gated residual connections

Abstract

The invention belongs to the technical field of text classification, and discloses a correlation false news detection method of adaptive gated residual connection. The invention uses Electra encoder to convert news documents into word embedding vectors; inputs the word embedding vectors into a multi-scale recurrent convolutional neural network to extract word embedding features of global or local semantics; and obtains keyword embedding features through adaptive gated residual connection; in batch samples, uses document-level cross attention to give news documents different influence weights according to the correlation between news documents, and obtains the output of document-level cross attention; and obtains the final document features through adaptive gated residual connection; the final document features are sent to a classifier, and the classifier outputs the news detection result. The invention can improve the accuracy and robustness of false news detection tasks by using adaptive gated residual connection and document-level cross attention.

Description

Technical Field

[0001] The present invention belongs to the technical field of text classification in natural language processing, and in particular relates to a correlation false news detection method of an adaptive gated residual connection. Background Art

[0002] In today's information age, the speed and scope of news dissemination are unprecedented, and people can obtain various information anytime and anywhere through the Internet. There may be some fake news in the news, which needs to be detected and filtered. Fake news detection technology based on text classification provides an effective solution. Text classification is a core technology in the field of natural language processing. It uses computer algorithms to automatically classify text content to identify topics, emotions, intentions, etc. Text classification technology is applied to fake news detection. According to some subtle differences in news materials in writing style and language features, emotions, sentiment features, content features, dissemination features and metadata features, fake news can be automatically identified, thereby improving the quality and credibility of information. In text classification technology, the quality of word embedding directly determines the final prediction performance, and good feature engineering is a prerequisite for achieving high performance. In the early days, static language models such as word embedding and global vector word representation were most commonly used. The disadvantage of these models is that they cannot express polysemous words. Later, large-scale dynamic language models based on self-attention mechanisms and pre-training modes emerged, such as Transformer-based Bidirectional Encoder Representation (BERT). BERT can achieve excellent natural language processing performance by pre-training in general domains and fine-tuning in combination with downstream tasks. Therefore, most of the existing fake news detection methods are based on models such as BERT as word embedding models.

[0003] Deep learning networks such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs) are commonly used feature extraction tools in text classification. Long short-term memory (LSTM) and gated recurrent unit (GRU) belong to the category of recurrent neural networks and are good at capturing contextual information from text, while TextCNN belongs to the category of convolutional neural networks and is good at extracting local feature information from text. Therefore, existing fake news detection methods usually use RNNs or CNNs to further extract feature information from text.

[0004] The self-attention mechanism, also known as the internal attention mechanism, is an important technology in deep learning, especially in the text classification task in the field of natural language processing. The core idea of ​​the self-attention mechanism is to enable the model to automatically learn the relationship between elements in sequence data (such as text) to capture richer contextual information. In order to focus the model's attention on the important parts of the news document during prediction, existing news detection methods often calculate the self-attention score of the news document before prediction.

[0005] There are literatures that disclose fake news detection methods based on machine learning, which use classic models such as SVM, random forest, and decision tree to predict fake news with relatively small resource consumption and faster speed. These methods show excellent performance on small-scale data sets. However, when encountering large-scale data sets or more complex tasks, these methods are often insufficient. Fake news detection methods based on machine learning usually use bag-of-words models and TF-IDF (Term Frequency-inverse Document Frequency) to extract text features. These statistical methods have difficulty extracting high-quality text features from complex news documents, which also lack semantic information. In addition, when processing longer news documents, it is difficult to capture potential information. In short, although some achievements have been made in the field of fake news detection, there is still room for improvement. Summary of the invention

[0006] The present invention proposes a correlation fake news detection method based on adaptive gated residual connections to improve the accuracy and robustness of fake news detection tasks.

[0007] The present invention is implemented by the following technical solution: a correlation false news detection method of adaptive gated residual connection, the steps are as follows:

[0008] S1: Use Electra encoder to convert news documents into word embedding vectors;

[0009] S2: input the word embedding vector into a multi-scale recurrent convolutional neural network to extract the word embedding features of global or local semantics; the word embedding vector controls the residual flow through an adaptive gated residual connection, and the obtained residual information and the word embedding features are superimposed by pixel addition to obtain the keyword embedding features; the multi-scale recurrent convolutional neural network (MSRCNN) is composed of multiple RCNNs in parallel;

[0010] S3: In the batch samples, the document-level cross attention is used to give different influence weights to news documents according to the correlation between them, and the output of the document-level cross attention is obtained; the document features obtained after averaging the word embedding features are controlled by adaptive gated residual connection to control the residual flow, and then superimposed with the output of the document-level cross attention by pixel addition to obtain the final document features;

[0011] S4: Send the final document features to the classifier, and the classifier outputs the news detection results.

[0012] Further preferably, the focal loss function is calculated based on the classifier output.

[0013] Further preferably, the operation of the multi-scale recurrent convolutional neural network is expressed by the following formula:

[0014] (1);

[0015] In the formula, X is the input sentence, is the word embedding matrix of Electra, is the word embedding vector obtained through Electra, Representation scale The circular convolution operation under It means merging word embedding features of different scales to obtain word embedding features E with global or local semantics.

[0016] Further preferably, in step S2, the process of obtaining the keyword embedding feature is expressed by the following formula:

[0017] (2);

[0018] (3);

[0019] (4);

[0020] In the formula, is the word embedding vector obtained through Electra The difference between the word embedding feature E output by the multi-scale recurrent convolutional neural network, The size of indicates the degree to which the multi-scale recurrent convolutional neural network extracts information; represents the weight of the gated switch of the adaptive gated residual connection of the word embedding vector obtained by Electra, is the parameter matrix of the adaptive gated residual connection of the word embedding vector obtained by Electra, and sigmoid is the activation function; Keyword embedding features output by adaptive gated residual connection.

[0021] Using document-level cross attention, different influence weights are given to news documents according to the relevance between them, and the output of document-level cross attention is obtained. The process is as follows:

[0022] Further optimization, if the news document has N keyword embedding features: , the document-level features obtained by averaging all keyword embedding features are: , Embed the features of the ith keyword in the news document; the document-level features of each news document form a document-level feature vector. After multiplying the document-level feature vector by a learnable randomly initialized parameter matrix, a query vector Q, a key vector K and a value vector V are generated. By calculating the dot product of the query vector Q of a news document and the key vector K of another news document, a relevance attention score matrix of each pair of news documents is obtained. After dividing by a scaling factor, it is normalized by the Softmax function to obtain the relevance attention weight of each news document with other news documents; finally, the value vector V of each news document is weighted and summed according to the relevance attention weights with other news documents to obtain the output of document-level cross attention.

[0023] Further preferably, the output calculation formula of the document-level cross attention is as follows:

[0024] (5);

[0025] In the formula, is the query vector of the mth news document, is the key vector of the j-th news document, for The transpose of is the value vector of the j-th news document, m≠j, is the scaling factor, Represents the output of document-level criss-cross attention.

[0026] Further preferably, the process of obtaining the final document features in step S3 is expressed as follows:

[0027] (6);

[0028] (7);

[0029] (8);

[0030] In the formula, The difference between the document feature D obtained by average pooling and the document-level cross-attention output feature F, the weight of the gate switch express The similarity score between the document-level cross-attention output feature F, is the parameter matrix, sigmoid is the activation function, The final document features.

[0031] The beneficial effects of the present invention are as follows:

[0032] In view of the fact that existing methods have ignored the mutual influence between news articles, have not considered the influence of the truthfulness of news on the truthfulness of related news, and have ignored the trust transfer phenomenon of news articles, the present invention introduces a document-level (rather than word-to-word) cross-attention mechanism to give documents different influence weights according to the relevance between news articles, thereby improving the performance of news detection.

[0033] In view of the traditional residual connection that directly adds previous information to the current information, transmitting too much previous information is likely to have a negative impact on the current feature information. The present invention proposes an adaptive gated residual connection module that adaptively balances the data flow of the original information and the residual information according to the sufficiency of the transmitted information, reduces unnecessary information transmission, and adaptively focuses on key features.

[0034] In order to solve the problems of global subtle feature extraction and sample imbalance in news detection, the present invention organically combines the efficient replacement word classification encoder (Electra), multi-scale recurrent convolutional neural network (MSRCNN), adaptive gated residual connection (AGRC), document-level cross attention mechanism, focal loss and other technologies, and effectively improves the performance of news detection by adaptively focusing on key features and document trust transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a model structure diagram of the present invention.

[0036] Figure 2 Schematic diagram of adaptive gated residual connection.

[0037] Figure 3 Schematic diagram of document-level cross attention.

[0038] Figure 4 Ablation experiment results on NewsHeadline dataset;

[0039] Figure 5 Ablation experiment results on the Covid dataset;

[0040] Figure 6 Ablation experiment results on the Politifact dataset;

[0041] Figure 7 Different for NewsHeadline The detection accuracy of the value;

[0042] Figure 8 Different for Covid The detection accuracy of the value;

[0043] Fig. 9 Different for Politifact The detection accuracy of the value. DETAILED DESCRIPTION

[0044] Reference Figure 1 , a correlation fake news detection method with adaptive gated residual connections, the steps are as follows:

[0045] S1: Remove stop words and other irrelevant symbols from the news document dataset, trim or pad sentences according to the average length of the news documents, and use the Electra encoder to convert the news documents into word embedding vectors;

[0046] S2: Input the word embedding vector into the multi-scale recurrent convolutional neural network (MSRCNN) to extract the word embedding features of global or local semantics; the word embedding vector controls the residual flow through the adaptive gated residual connection, and the residual information obtained is superimposed with the word embedding features by pixel addition to obtain the keyword embedding features;

[0047] S3: In the batch samples, the document-level cross attention is used to give different influence weights to news documents according to the correlation between news documents, and the output of the document-level cross attention is obtained; the document features obtained after keyword embedding feature pooling are controlled by adaptive gated residual connection to control the residual flow, and then superimposed with the output of the document-level cross attention by pixel addition to obtain the final document features;

[0048] S4: The final document features are fed into the classifier, which outputs the news detection results. The focal loss function is calculated based on the classifier output to balance the sample data volume and improve the classification performance of difficult-to-classify samples.

[0049] The present invention applies the Electra encoder to extract word embedding vectors. The architecture of the Electra encoder is similar to BERT, and it consists of a small masked language model called a generator and a discriminator. The generator is responsible for generating the replaced vocabulary, while the discriminator distinguishes between real vocabulary and generated alternative vocabulary. Through this structure, the Electra encoder can effectively use a large-scale corpus for pre-training, and has high computational efficiency. It has now become one of the outstanding language models in the field of natural language processing. For this reason, the present invention selects the Electra encoder as the language model for news documents. After obtaining the word embedding representation through the Electra encoder, the news document is further input into the multi-scale recurrent convolutional neural network (MSRCNN) to extract contextual features.

[0050] In order to extract subtle global or local semantic features, the present invention introduces a multi-scale recurrent convolutional neural network (MSRCNN). The multi-scale recurrent convolutional neural network consists of three independent RCNNs. RCNN is a model based on convolutional neural networks and recurrent neural networks. It combines the advantages of CNN in capturing local features and the advantages of RNN in modeling sequence information. RCNN consists of a bidirectional long short-term memory network (Bidirectional Long Short-Term Memory, BiLSTM) sub-block and a maximum pooling MaxPool1d sub-block. BiLSTM introduces an RNN-based gating mechanism that can adaptively adjust the propagation of sequence information, which can promote the model to extract deep contextual information and global information from news documents. The maximum pooling MaxPool1d sub-block is used to downsample the input data, thereby reducing the dimensionality and complexity of the data while retaining important feature information, and can extract local feature information from news documents. The multi-scale recurrent convolutional neural network (MSRCNN) connects multiple RCNNs in parallel, aligns the output vector of RCNN with the overall hidden layer size of the model, and allows the model to obtain contextual information from news documents at different spatial and scale levels, which helps the model to more comprehensively understand the deep information in the news documents. The operation of the multi-scale recurrent convolutional neural network can be expressed by the following formula:

[0051] (1);

[0052] In the formula, X is the input sentence, is the word embedding matrix of Electra, is the word embedding vector obtained through Electra, Representation scale The circular convolution operation under It means merging word embedding features of different scales to obtain word embedding features E with global or local semantics.

[0053] In order to deal with the gradient vanishing problem, people usually introduce residual skip connections into the model. However, the traditional residual skip connection directly adds the previous information to the current information. Too much previous information may have a negative impact on the current feature information. Therefore, the present invention proposes an adaptive gated residual connection, such as Figure 2 As shown in the figure, the gating mechanism is used to adaptively balance the data flow of the original information and the residual information, reducing unnecessary information transmission. The traditional residual skip connection directly adds the previous information to the current information without considering whether the current network needs to receive too much previous information, while the adaptive gated residual connection controls the reception of the residual information by adding a gate switch, which helps the model focus on more important features.

[0054] The weight of the gate switch is generated according to the correlation between the residual information and the current information. The specific calculation formula is as follows:

[0055] (2);

[0056] (3);

[0057] In the formula, is the word embedding vector obtained through Electra The difference between the word embedding feature E output by the multi-scale recurrent convolutional neural network, The size of indicates the degree to which the multi-scale recurrent convolutional neural network extracts information; represents the weight of the gated switch of the adaptive gated residual connection of the word embedding vector obtained by Electra, is the parameter matrix of the adaptive gated residual connection of the word embedding vector obtained by Electra, sigmoid is the activation function; the residual information Concatenate with word embedding feature E to obtain the correlation value between residual information and current information, and then multiply by , get the similarity score, this similarity score is passed through a sigmoid activation function to get a probability value from 0 to 1, that is, the weight of the gate switch Then the flow of residual information is controlled by the weight of the gate switch, and the calculation formula is as follows:

[0058] (4);

[0059] In the formula, Keyword embedding features output by adaptive gated residual connection.

[0060] In the real world, evaluating the authenticity of a piece of news usually requires comparison with other related news articles. If there are many news articles similar to a piece of news that are proven to be true, then the content of the news is likely to be true; conversely, if there are many news articles that are contrary to a piece of news that are proven to be true, then the content of the news is likely to be false. In other words, the trust of a news article can be transferred. However, existing methods have ignored the trust transferability between news articles and have not considered the fact that the truthfulness of news is often affected by the truthfulness of related news. The cross-attention used in existing fake news detection is usually used to calculate the correlation between words, ignoring the role of the cross-attention mechanism in calculating the correlation between news articles. To this end, the present invention introduces a cross-attention mechanism at the document level (rather than between words), and gives different influence weights to documents according to the correlation between news articles to improve the performance of news detection.

[0061] Figure 3 It is a schematic diagram of document-level cross attention. Figure 3 Each square in the data (such as "News 1", "News 2") represents a news document in the batch. These news documents are related to each other, and the dotted lines represent the cross-attention weights between different documents. When training the model, each training document is processed by Electra and MSRCNN to extract the keyword embedding features of the news document, and then average pooling is used to average all the keyword embedding features in the news document to obtain the feature vector representing the overall news document. If a news document has N keyword embedding features: , the document-level features obtained by averaging all keyword embedding features are: , is the i-th keyword embedding feature of the news document. Subsequently, the document-level features are used to calculate the relevance of each news document in the batch to obtain the relevance attention weight. Specifically, the document-level features of each news document form a document-level feature vector. After multiplying the document-level feature vector by a learnable randomly initialized parameter matrix, a query vector Q, a key vector K, and a value vector V are generated. By calculating the dot product of the query vector Q of a news document and the key vector K of another news document, the relevance attention score matrix of each pair of news documents is obtained. After dividing by a scaling factor, it is normalized by the Softmax function to obtain the relevance attention weight of each news document with other news documents. Finally, the value vector V of each news document is weighted and summed according to the relevance attention weight of other news documents to obtain the output of document-level cross attention. The specific calculation formula is shown in formula (5).

[0062] (5);

[0063] In the formula, is the query vector of the mth news document, is the key vector of the j-th news document, for The transpose of is the value vector of the j-th news document, m≠j, is the scaling factor, Represents the output of document-level criss-cross attention.

[0064] When testing the model, there is a news document in a batch that is a news document to be tested, while the other news documents are news documents with known labels. When testing the model, it is necessary to obtain the weighted output of the news document to be tested based on the cross-attention between the news document to be tested and all the news documents with known labels in the batch. Therefore, it is also necessary to use average pooling for all news documents in the batch to average all keyword embedding features in the news document to obtain the feature vector of the overall representation of the news document, and then multiply the news document to be tested by the learned parameter matrix to obtain the query vector (Q), and multiply each news document with known labels by the learned parameter matrix to obtain the key vector (K) and value vector (V). According to formula (5), the cross-attention between the document to be tested and all the news documents with known labels in the batch is calculated. Finally, the relevance attention weight is weighted and summed on the value vector V of the news document with known labels to obtain the weighted output of the news document to be tested.

[0065] Document-level cross-attention can fuse the information of different news documents together through weighted summation to generate an overall representation, which can convey the trust between news documents and capture the mutual influence between news documents, thereby improving the accuracy of fake news detection. The document-level cross-attention mechanism is used to focus on the relationship between different news documents. The present invention also introduces a self-attention mechanism to focus on the lexical relationship within a single news document. This multiple attention mechanism can help the model better understand and evaluate the authenticity of news.

[0066] In this embodiment, the document feature controls the residual flow through an adaptive gated residual connection, and then superimposes the document-level cross-attention output by pixel addition to obtain the final document feature; it is expressed as follows:

[0067] (6);

[0068] (7);

[0069] (8);

[0070] In the formula, The difference between the document feature D obtained by average pooling and the document-level cross-attention output feature F, the weight of the gate switch express The similarity score between the document-level cross-attention output feature F, is the parameter matrix, sigmoid is the activation function, The final document features.

[0071] In order to deal with sample imbalance and improve the classification performance of difficult-to-classify samples, the present invention introduces a focal loss function to replace the conventional cross entropy loss function. The focal loss function adjusts the relationship between multiple samples and few samples by introducing two hyperparameters, and its calculation method is shown in formula (9).

[0072] (9);

[0073] In the formula, is the focal loss, represents the predicted probability of the positive class, It is an adjustment factor used to reduce the attention paid to samples that are easy to classify and increase the attention paid to samples that are difficult to classify. is the logarithm of the predicted probability and is a component of the cross entropy loss. It is a parameter that adjusts the weights of positive and negative samples to alleviate the problem of class imbalance.

[0074] The present invention conducts experiments on three public fake news detection datasets: NewsHeadline, Covid, and Politifact. The detailed information of the datasets is shown in Table 1.

[0075] Table 1 Dataset information

[0076]

[0077] As can be seen from Table 1, the fake news detection dataset used in the present invention includes both long texts and short texts. Therefore, selecting NewsHeadline, Covid and Politifact can better summarize the field of fake news detection. Table 2 is a dataset sample distribution table, which lists the number of real news and fake news in each dataset. As can be seen from Table 2, datasets for detecting fake news generally have the problem of sample imbalance. Positive samples account for the majority, and negative samples account for the minority. The introduction of focal loss can alleviate this problem, thereby improving the prediction accuracy of the fake news detection task.

[0078] Table 2 Dataset sample distribution table Unit: %

[0079]

[0080] In the experiment of this invention, the learning rate is 5e-6 and the batch size is 64. The default training runs for 30 epochs, and the training is stopped early if the loss on the validation set does not decrease for 2000 consecutive batches. The length of sentences in the dataset is clipped according to their average length. Adam is selected as the optimizer with a dropout rate of 0.1. .

[0081] News detection belongs to the category of text classification. Therefore, the present invention selects the commonly used accuracy, precision, recall and F1 score in text classification as the evaluation indicators of the experiment. The calculation formula is as follows:

[0082] (10);

[0083] (11);

[0084] (12);

[0085] (13);

[0086] Wherein, TP represents true positive, TN represents true negative, FP represents false positive, and FN represents false negative. In addition, in order to better evaluate the performance on the imbalanced data set, the present invention adopts the Macro method to calculate the F1 score.

[0087] In order to verify the performance of the method proposed in the present invention in the task of detecting fake news, the present invention conducted a comparative experiment.

[0088] Table 3 Classification performance on each data set (%)

[0089]

[0090] The methods involved in the comparison are: Naive Bayes (NB), Multilayer Perceptron (MLP) and k Nearest Neighbor (KNN), Convolutional Neural Network (CNN), Bidirectional Long Short-Term Memory Network (BiLSTM), Graph Convolutional Network (GCN), Fast Text Classification (FastText), Bidirectional Encoder Representation (BERT) and Robustly Optimized BERT Pretraining Approach (RoBERTa), as well as some methods most relevant to the present invention: Bi-LSTM-GRU, CNN-RNN and Word2Vec+LSTM. These methods are classic or current newer classification methods in the field of text classification. Among them, NB, MLP and KNN use TF-IDF (Term Frequency- inverse Document Frequency) to uniformly convert news documents into feature vectors, and CNN and BiLSTM use GloVe (Global Vectors for Word Representation) word embedding. The pre-trained model files loaded by BERT and RoBERTa are from Huggingface.

[0091] The results of the comparative experiment are shown in Table 3. It can be seen from the table that the method proposed in the present invention has achieved the highest classification accuracy, recall rate and F1 score on the three data sets (NewsHeadline, Covid and Politifact). This shows that the method of the present invention performs best in the fake news detection task and has significant advantages over other methods. This is mainly because the Electra encoder used in the present invention has a text feature extraction capability that is superior to Word2Vec and BERT; the adaptive gated residual connection introduced in the present invention effectively reduces the risk of overfitting and enhances the feature expression capability; the focal loss function used in the present invention can effectively solve the problem of unbalanced data sets.

[0092] Compared with traditional machine learning methods, deep learning methods (such as CNN, BiLSTM, GCN, BERT, RoBERTa, etc.) generally perform better in most cases. This shows that for fake news detection tasks, deep learning methods can more effectively capture features and semantic information, thereby improving classification performance.

[0093] It is worth noting that for small-scale datasets, traditional machine learning methods have achieved better predictive performance than deep learning methods. For example, in the Covid dataset, MLP achieved an accuracy of 94.49%, which is better than most deep learning methods. The superior performance of traditional machine learning methods on small-scale datasets may be due to the fact that the features of these datasets are more suitable for traditional machine learning algorithms. It is possible that the features of these datasets have good linear separability or traditional machine learning algorithms are better at capturing these features. However, when the size of the dataset increases and the task becomes more complex, traditional machine learning methods may fall short. This is because, in the context of large-scale datasets and complex tasks, the data often exhibits more features and more complex relationships, which traditional machine learning methods may not be able to effectively capture. In contrast, deep learning methods can better handle complex data representations through hierarchical feature learning.

[0094] In order to further verify the effectiveness of each module of the proposed method, an ablation experiment was carried out. Figure 4-Figure 6 The “+” symbol in the figure indicates that the current module is introduced based on the previous module.

[0095] When comparing the Electra encoder with BERT, the prediction accuracy of the Electra encoder is significantly higher than that of BERT. This clearly demonstrates the effectiveness of the proposed method in utilizing the Electra encoder as a language model, and the Electra encoder is able to generate higher quality vectors for news documents.

[0096] Introducing focal loss (+FL) on top of the Electra encoder improves the accuracy of fake news detection tasks by 1.23%, 0.42%, and 0.75% on the three datasets, respectively. This clearly demonstrates the effectiveness of focal loss in addressing the problem of dataset sample imbalance. Focal loss increases the weight of minority samples in the loss function to enhance the model's attention to these minority samples during training, thereby improving the overall prediction accuracy.

[0097] After the introduction of the multi-scale recurrent convolutional neural network (+MSRCNN), the accuracy was further improved by 0.24%, 0.25%, and 0.19% respectively on the basis of the previous accuracy. This shows that the multi-scale recurrent convolutional neural network can extract contextual information embedded in news documents and enhance the model's understanding of news content.

[0098] After introducing the adaptive gated residual connection (+AGRC), the accuracy was improved by 0.26%, 0.32%, and 0.29% respectively on the basis of the previous accuracy. This shows that the gated residual connection can adaptively adjust the degree to which residual information needs to pass, thereby avoiding potential information forgetting in the model. The required residual information is adaptively adjusted through a learnable parameter matrix to prevent the impact of excessive residual information on model performance.

[0099] After introducing the document-level cross-attention mechanism (+ATT), the accuracy was improved by 0.3%, 0.24%, and 0.18% respectively. This explains the calculation of the attention score between news, allowing the model to fully consider the relationship between news. Before judging the authenticity of the current news, it will fully consider the information from other news.

[0100] In summary, Figures 4 to 6 The experimental results show that the proposed method is effective in architecture and each module helps to improve the performance of the fake news detection task.

[0101] To determine the focal loss The optimal value of the present invention is different The experimental analysis was carried out under the value of The impact on prediction accuracy, experimental results are as follows Figure 7-Figure 9 shown. The value refers to the weight of the minority sample. The experimental results show that the experimental accuracy is There is a strong correlation between the values. On the three data sets, the highest accuracy The values ​​are 0.9, 0.5, and 0.6, respectively, which are related to their actual sample distribution. When the value is , the accuracy of the fake news detection task begins to improve, but too high The value will also lead to a decrease in accuracy.

[0102] Finally, it is determined that in the NewsHeadline dataset, the focal loss function The best performance is achieved when the value is 0.9, with an accuracy of 90.53%. The best performance is achieved when the value is 0.5, with an accuracy of 94.81%. The best performance was achieved when the value was set to 0.6, reaching 78.75%.

[0103] The invention described above only expresses the implementation methods of the embodiments of the present invention, and cannot be understood as limiting the scope of the invention patent, nor does it impose any form of limitation on the structure of the embodiments of the present invention. It should be pointed out that for ordinary technicians in this field, several changes and improvements can be made without departing from the concept of the embodiments of the present invention, which all belong to the protection scope of the embodiments of the present invention.

Claims

1. A correlation fake news detection method based on adaptive gated residual connection, characterized in that: The steps are as follows: S1: Use Electra encoder to convert news documents into word embedding vectors; S2: Input the word embedding vector into a multi-scale recurrent convolutional neural network to extract the word embedding features of global or local semantics; The word embedding vector controls the residual flow through an adaptive gated residual connection, and the obtained residual information and the word embedding feature are superimposed by pixel addition to obtain the keyword embedding feature; the multi-scale recurrent convolutional neural network is formed by connecting multiple RCNNs in parallel; the adaptive gated residual connection is to add a gate switch on the basis of the residual connection to control the reception of residual information; S3: In the batch samples, the document-level cross attention is used to give different influence weights to news documents according to the correlation between them, and the output of the document-level cross attention is obtained; the document features obtained after averaging the word embedding features are controlled by adaptive gated residual connection to control the residual flow, and then superimposed with the output of the document-level cross attention by pixel addition to obtain the final document features; S4: Send the final document features to the classifier, and the classifier outputs the news detection results.

2. According to claim 1, a correlation false news detection method based on adaptive gated residual connection is characterized in that: Calculate the focal loss function based on the classifier output.

3. The method for detecting false news with correlation by adaptive gated residual connection according to claim 1, characterized in that: The operation of the multi-scale recurrent convolutional neural network is expressed by the following formula: (1)； In the formula, X is the input sentence, is the word embedding matrix of Electra, is the word embedding vector obtained through Electra, Representation scale The circular convolution operation under It means merging word embedding features of different scales to obtain word embedding features E with global or local semantics.

4. The method for detecting false news with correlation by adaptive gated residual connection according to claim 3, characterized in that: In step S2, the process of obtaining keyword embedding features is expressed by the following formula: (2)； (3)； (4)； In the formula, is the word embedding vector obtained through Electra The difference between the word embedding feature E output by the multi-scale recurrent convolutional neural network, The size of indicates the degree to which the multi-scale recurrent convolutional neural network extracts information; represents the weight of the gated switch of the adaptive gated residual connection of the word embedding vector obtained by Electra, is the parameter matrix of the adaptive gated residual connection of the word embedding vector obtained by Electra, and sigmoid is the activation function; Keyword embedding features output by adaptive gated residual connection.

5. The method for detecting false news with correlation by adaptive gated residual connection according to claim 4, characterized in that: In step S3, using document-level cross attention, different influence weights are given to news documents according to the relevance between news documents to obtain the output of document-level cross attention. The process is as follows: If a news document has N keyword embedding features: , the document-level features obtained by averaging all keyword embedding features are: , Embed features for the i-th keyword of the news document; The document-level features of each news document constitute a document-level feature vector. After multiplying the document-level feature vector by a learnable randomly initialized parameter matrix, a query vector Q, a key vector K and a value vector V are generated. By calculating the dot product of the query vector Q of a news document and the key vector K of another news document, the relevance attention score matrix of each pair of news documents is obtained. After dividing by a scaling factor, it is normalized through the Softmax function to obtain the relevance attention weight of each news document with other news documents. Finally, the value vector V of each news document is weighted and summed according to the relevance attention weights with other news documents to obtain the output of document-level cross attention.

6. The method for detecting false news with correlation by adaptive gated residual connection according to claim 5, characterized in that: The output calculation formula of document-level cross attention is as follows: (5)； In the formula, is the query vector of the mth news document, is the key vector of the j-th news document, for The transpose of is the value vector of the j-th news document, m≠j, is the scaling factor, Represents the output of document-level criss-cross attention.

7. The method for detecting false news with correlation by adaptive gated residual connection according to claim 6, characterized in that: The process of obtaining the final document features in step S3 is as follows: (6)； (7)； (8)； In the formula, The difference between the document feature D obtained by average pooling and the document-level cross-attention output feature F, the weight of the gate switch express The similarity score between the document-level cross-attention output feature F, is the parameter matrix, sigmoid is the activation function, The final document features.

Images