Rumor detection method using self-attention generator and BiLSTM discriminator

Abstract

The present invention discloses a rumor detection method using a self-attention generator and a BiLSTM discriminator, comprising the following steps: collecting rumor text data to form a rumor dataset; constructing a generative adversarial network generator including a self-attention layer based on a self-attention mechanism; constructing a discriminator network to perform rumor detection and classification on the original rumor text and the text decoded by the generator; training the generative adversarial network to adjust the model parameters of the generator and the model parameters of the discriminator; extracting the discriminator network of the generative adversarial network to perform rumor detection on the text to be detected. The rumor detection method of the present invention has high detection accuracy and good robustness. The self-attention generator is used to construct key features through semantic learning of rumor samples, and generate text samples rich in performance features to simulate the information loss and confusion in the rumor propagation process. The discriminator's semantic feature recognition ability is enhanced through adversarial training.

Description

Technical Field

[0001] The present invention belongs to the field of text recognition, and in particular relates to a rumor detection method using a self-attention generator and a BiLSTM discriminator. Background Art

[0002] Rumors have temporal characteristics and diverse features. Information will be continuously processed during the dissemination process, making it very confusing.

[0003] Rumor detection algorithms based on traditional machine learning require the pre-definition and extraction of text-related features, and are highly dependent on specific datasets. Consequently, these algorithms have poor generalization capabilities. Neural network structures such as LSTM, GRU, and CNN are used to extract rumor features, avoiding the need for artificial feature engineering. However, these models fail to incorporate the characteristics of rumor propagation within networks, making it difficult to discern key semantic features. Furthermore, in real life, rumors often change marginal information over time, increasing semantic confusion and reducing the model's ability to extract key features, resulting in suboptimal detection results.

[0004] In summary, traditional rumor detection algorithms have problems such as unsatisfactory effects in extracting text semantics and key features, while general sequence models cannot solve the problem of feature extraction under specific semantics in text detection, resulting in poor model generalization ability. Summary of the Invention

[0005] The purpose of the present invention is to address the above-mentioned problems and provide a rumor detection method, which adopts a rumor detection model that combines a self-attention mechanism with a generative adversarial network. The model adopts an adversarial training method, which conforms to the characteristics of rumors that artificially add, delete, exaggerate and distort information during the propagation process. Through the mutual promotion of the adversarial network generator and discriminator, the learning of the indicative features of rumors is strengthened, and the learning ability of the model is continuously improved; the generator adopts a self-attention layer to realize semantic extraction and feature learning; the discriminator adopts a bidirectional LSTM to improve the ability to judge long-sequence rumors.

[0006] The technical solution of the present invention is a rumor detection method using a self-attention generator and a BiLSTM discriminator, comprising the following steps:

[0007] Step 1: Collect rumor text data to form a rumor dataset;

[0008] Step 2: Based on the self-attention mechanism, build a generative adversarial network generator containing a self-attention layer;

[0009] Step 3: Build a discriminator network to detect and classify rumors on the original rumor text and the text decoded by the generator;

[0010] Step 4: Train the generative adversarial network and adjust the model parameters of the generator and the discriminator;

[0011] Step 5: Extract the discriminator network of the generative adversarial network and perform rumor detection on the text to be detected.

[0012] Furthermore, the rumor detection model includes a data preprocessing part, a generator, a discriminator network and an output layer.

[0013] Preferably, the generator includes a preprocessing layer, multiple fully connected layers, multiple self-attention layers, a residual structure and a classifier, and the jump connection directly connects the multiple self-attention layers and the fully connected layers to form a residual unit.

[0014] Preferably, the discriminator is a BiLSTM network, comprising a plurality of forward LSTM units and a plurality of reverse LSTM units.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1) This paper adopts a rumor detection model based on a self-attention generator and a BiLSTM discriminator. Compared with existing rumor detection methods, it has higher detection accuracy and better robustness.

[0017] 2) The generator of the present invention uses a self-attention layer to construct key features through semantic learning of rumor samples, generating text samples rich in expressive features to simulate the information loss and confusion during the rumor propagation process, and enhances the semantic feature recognition ability of the discriminator through adversarial training;

[0018] 3) The rumor detection model of the present invention adopts a multi-head self-attention structure, which can realize parallel computing of the network and effectively reduce the time complexity of network training;

[0019] 4) The rumor detection model of the present invention has strong generalization capabilities and is effective in detecting rumors in Chinese and foreign language texts;

[0020] 5) The rumor detection method of the present invention adopts a generative adversarial training approach, which allows the rumor detection model to be trained with a small number of rumor samples, thus avoiding the defects of existing rumor detection models in that the training requires a large number of rumor samples and the rumor samples are difficult to collect;

[0021] 6) The present invention uses the GloVe algorithm to extract word vectors from texts. This algorithm takes a short time to train word vectors and the trained word vectors are accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention will be further described below with reference to the accompanying drawings and examples.

[0023] Figure 1Schematic diagram of the structure of the rumor detection model according to an embodiment of the present invention.

[0024] Figure 2 Schematic diagram of the structure of the generator of an embodiment of the present invention.

[0025] Figure 3 Schematic diagram of the structure of the discriminator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] use Figure 1 The rumor detection model shown in the figure uses a self-attention generator and a BiLSTM discriminator to detect rumors. It includes the following steps that are performed in sequence:

[0027] Step 1: Collect rumor text data to form a rumor dataset; preprocess the text data, filter the text data for special symbols, and vectorize the words;

[0028] Step 2: Construct a generative adversarial network generator with a self-attention layer to encode and decode the original text, such as Figure 2 As shown, the feature distribution is biased towards the opposing category as much as possible, and the difference between the original sequence and the disguised sequence is maximized, thereby strengthening the discriminator model's ability to detect rumors;

[0029] Step 3: Construct a generative adversarial network discriminator, such as Figure 3 As shown, rumor detection is performed on the original text and the text sequence decoded by the generator, and the text with the same label is classified;

[0030] Step 4: Use the loss value generated by the discriminator to perform generative adversarial training, adjusting the parameters of the generator network model and the discriminator network model in batches to improve the accuracy of rumor detection;

[0031] Step 5: Extract the discriminator of the generative adversarial model to detect rumors on the text to be detected and determine the generalization ability of the model.

[0032] like Figure 1 As shown in the figure, the rumor detection model includes data preprocessing, generator network, discriminator network and output layer.

[0033] like Figure 2 As shown in the figure, the generator includes a preprocessing layer, multiple fully connected layers, a self-attention layer, a residual structure and a classifier. The skip connection directly connects multiple self-attention layers and fully connected layers to form a residual unit.

[0034] like Figure 3 As shown in the figure, the discriminator is a BiLSTM network, which includes multiple forward LSTM units and reverse LSTM units.

[0035] In step 3, rumor detection is performed on the original rumor text and the text decoded by the generator, specifically including:

[0036] 1) The location information of the rumor text is embedded into the word vector. For the short rumor text sequences in the dataset, the data is cleaned of special symbols, segmented, and filtered by word frequency.

[0037] Use the GloVe algorithm to vectorize the word and define X ij represents the total number of times word j appears in the context of word i; X i represents the total number of times all words appear in the context of word i; P ij =P(j|i)=X ij / X i , represents the probability that word j appears in the context of word i. Define function f(x ij ) represents X ij The weight in the text is calculated as follows:

[0038]

[0039] Where x max Set to 100, α to 0.75;

[0040] The objective function J is defined as follows:

[0041]

[0042] Where n represents the number of words in the dataset, w i , w k Represents the word vector of word i, j, b i , b k Indicates the word bias introduced. The GloVe algorithm finds the minimum value of the objective function J and obtains the word vector w j , j = 1, 2…, n, and then get the sequence S consisting of word vectors i ={w1, ..., w n}, i represents the specific position of each word vector; these n word groups represent all the information of this sequence. The position information of the word in the sequence is marked as pos, and the position information is encoded using trigonometric functions:

[0043]

[0044] where d model is the dimension of the word in the sequence, i represents the position of the word vector, x (pos，i) Represents the position code of the i-th word at the pos position, and then calculates a set of position information vectors wp i ={x1, ..., x n}, combined with the word semantic vector S i , the final input to the generator or discriminator is the word vector M i =S i +wp i ;

[0045] The GloVe algorithm used in the embodiment refers to the conference paper “Glove: Global Vectors for Word Representation” published by Pennington J et al. in 2014.

[0046] 2) The word vector obtained by preprocessing is input into the generator network, where the self-attention layer extracts its features. First, the three state matrices W are initialized q 、W k 、W v , which facilitates subsequent parallel attention weight calculation:

[0047] x q =M i W q

[0048] x k =M i W k

[0049] x v =M i W v

[0050] By calculating the vector x q 、x k 、x v To represent M i , that is, M i ={x q , x k , x v}; Enter the self-attention layer and perform matrix operations on the q and k sub-vectors of each word vector in turn to obtain the score:

[0051] score i =x q ·x k

[0052] Normalize and input into the softmax function to get the output vector c of the current word vector for the rumor sequence i The conversion weight and attention calculation formula are as follows:

[0053]

[0054] a(scorei , x v ) i Represents the word vector M i For the output vector c i The contribution of n is the length of the word, and the value range is n∈[1,512];

[0055] After the self-attention layer, it enters the fully connected layer for feature extraction. r and non-rumor X n The generator constructs the following nonlinear function to map the original features into the new feature space, that is:

[0056]

[0057] 3) The original text sequence and the text sequence generated by the generator are input into the discriminator, which is composed of LSTM structural units, such as Figure 3 As shown in Figure 2, first, the forward and reverse LSTM layer states are initialized with a0=(h0, ce0), where h0 is the LSTM explicit state, ce0 is the LSTM unit cell state, and the hidden state a0 is randomly initialized using a normal distribution. i ={x1, ..., x n} Enter the LSTM unit of the discriminator, and the input gate inside the LSTM unit extracts information from the sequence:

[0058] i t =σ(W i ·[h t-1 , x t ]+b i )

[0059] C t =tanh(W c ·[h t-1 , x t ]+b c )

[0060] Where h t-1 is the previous time series state, x t is the input at the current moment t, and is also the rumor sequence word vector processed by the self-attention layer. t is the semantic information of the input word at the current moment and the output at the previous moment, W i , W c , W c , b i is the parsing matrix of the rumor sequence by the input gate. In order to filter out some feature information irrelevant to classification and retain key features, C t , x tInput to the forget gate and perform the following feature filtering calculations:

[0061] f t =σ(W f ·[h t-1 , x t ]+b f )

[0062] Where W f , b f are the weight matrices of the forget gate respectively. After information extraction and filtering, the new word information is calculated

[0063]

[0064] Get the feature matrix to be output, LSTM controls this state information and input feature vector to output o at the current moment t The influence of , then enters the output gate and finally obtains the time sequence rumor word vector ρ t ;

[0065] o i =σ(W o [h i-1 , x i ]+b o )

[0066]

[0067]

[0068] Where W o , b o are the weight matrices of the output gate, is the sequence vector obtained by LSTM forward or backward, is the sequence vector obtained by LSTM forward, is the sequence vector obtained by LSTM reverse. The last layer of the discriminator network obtains the output T of all time series i ={ρ i |0<i<n}, considering that rumor detection should be affected by key features rather than some marginal features, an attention layer is introduced at the output layer of the discriminator network. In the input sequence, the feature matrix is ​​constructed For each LSTM unit output ρ i Calculate the impact factor e i :

[0069]

[0070]

[0071] After normalization, the weight of each time series is obtained, and then the weighted sum of each time series is calculated according to the weight:

[0072]

[0073] Finally, the softmax layer is used to calculate the category probabilities of rumors and non-rumors.

[0074] In step 4, when training the generative adversarial network, the generator decodes the text sequence into opposing category text as much as possible, defines the mapping function f(X), and establishes a representation mechanism for rumors disguised as non-rumors and non-rumors disguised as rumors. However, a single mapping relationship may cause the sequence to be over-decoded, and the original semantics may be changed to the target semantics, thereby reducing the classification ability of the discriminator. To this end, it is necessary to change the mapping function of the generator to be reversible, that is, to use the output of the generator network model as its input to ensure that the original input is obtained. Define the mapping relationship:

[0075]

[0076] The reversibility of the generator ensures that the decoded features can restore the original features and generate a chaotic feature sequence to achieve the purpose of the generator. The loss function of the generator is composed of the predicted classification difference f based on the original sequence. s (X), predicted classification difference based on generated sequence And their Euclidean distance composition:

[0077]

[0078] Where D represents the calculation function of the classification difference of the discriminator model. The discriminator network uses the BiLSTM model, and its loss value is composed of the original classification difference and the discriminant value of the sequence generated by the generator:

[0079] L D =D(X)+D(f S (X)

[0080] As a generative adversarial network model for classification, the goal is for the generator to disguise rumor and non-rumor sequences, skewing their feature distribution as much as possible toward the opposing category and maximizing the difference between the original and disguised sequences. The discriminator, on the other hand, needs to train its network to distinguish the category of the original sequence and strive to classify the disguised and original sequences into the same category, maximizing their commonality. These features are the key features that the discriminator needs to learn. The optimization model for the adversarial network is as follows:

[0081] V(G, D)=αL G +(1-α)L D

[0082]

[0083] Where α is the loss coefficient in the weighted adversarial network, the default value is 0.5, and V represents the model's loss function L for the generator. G and the discriminator loss function L D The relationship mapping, G * Indicates the model optimization method.

[0084] When training a generative adversarial network, the discriminator parameters are first updated to maximize the difference between the generator's disguised sequence and the desired target sequence. The generator's network structure and parameters are then optimized so that the sequence generated by the generator approximately matches the semantics of the desired target sequence, preventing the discriminator in the adversarial network from distinguishing the characteristic differences between rumor-disguised sequences and non-rumors.

[0085] The implementation results show that: compared with the existing rumor detection methods, the rumor detection method of the present invention has higher detection accuracy and better robustness; the rumor detection model of the present invention has strong generalization ability and has excellent rumor detection effect on both Chinese text and foreign language text; using the generative adversarial training method, a small number of rumor samples can complete the training of the rumor detection model; using the GloVe algorithm to extract the word vector of the text, the GloVe model takes a short time to train the word vector, the trained word vector is accurate, and the context information collection of word relationships and the word weight control are optimized compared with the CBOW model and the Skip-Gram model.

Claims

1. The rumor detection method using self-attention generator and BiLSTM discriminator is characterized by: The generator includes a preprocessing layer, an encoder, and a decoder connected in sequence. The preprocessing layer uses the GloVe algorithm to vectorize the input words; the encoder includes a self-attention layer, a residual structure, and a forward fully connected layer connected in sequence; the decoder includes a self-attention layer, a residual structure, a forward fully connected layer, an inter-layer attention layer, and a forward fully connected layer connected in sequence. The self-attention layer and the inter-layer attention layer of the decoder are respectively connected to the forward fully connected layer of the encoder, and the self-attention layer and the fully connected layer are directly connected front and back using skip connections to form a residual unit; The method comprises the following steps, Step 1: Collect rumor text data to form a rumor dataset; Step 2: Based on the self-attention mechanism, build a generative adversarial network generator containing a self-attention layer; Step 3: Build a discriminator network to detect and classify rumors on the original rumor text and the text decoded by the generator; Step 4: Train the generative adversarial network and adjust the model parameters of the generator and the discriminator; Step 5: Extract the discriminator network of the generative adversarial network and perform rumor detection on the text to be detected.

2. The rumor detection method according to claim 1, characterized in that: The rumor detection model includes a data preprocessing part, a generator, a discriminator network and an output layer.

3. The rumor detection method according to claim 1, wherein: The discriminator is a BiLSTM network, which includes multiple forward LSTM units and multiple reverse LSTM units.

Images