A user stance detection method based on external common sense knowledge and deep learning

Abstract

The application provides a user stand detection method based on external common sense knowledge and deep learning, comprising the following steps: step 1, defining the user stand according to the tweets sent by the user in the social network; step 2, pre-processing the social network user data; step 3, extracting emotional information by using external common sense knowledge; step 4, extracting theme information; step 5, extracting neighbor text information; and step 6, attention mechanism of user historical information. The application sets a complementary gate mechanism to better obtain the emotional information vector extracted by the external common sense knowledge, trains different base classifiers through different emotional features, and then obtains a meta-classifier with a global view by using a local fusion strategy. The outputs of the meta-classifier are integrated through an accuracy-based weighted voting scheme, and the emotional roles in the social network are recognized, so that the recognition of the emotional roles is more effective.

Description

Technical Field

[0001] This invention provides a user stance detection method based on external common sense knowledge and deep learning, specifically relating to the field of information analysis technology. Background Technology

[0002] With the advancement of communication and internet technologies, social media platforms have provided diversified channels for information dissemination. Social media platforms constitute an important part of personal social interaction, allowing users to share their thoughts and engage in fun exchanges through text, images, videos, and emoticons via simple and diverse operation methods. Therefore, they have become an important source for people to understand current events, connect with the world, and obtain real-time information. They allow people of different ages to discuss emerging topics across various fields, express their opinions promptly, and receive feedback, enabling an event to spread widely and become a hot topic in a very short time. At the same time, discussions about a particular event on social networks are filled with various comments and opinions, which influence users' judgments and stances on the event itself. Behind every trending topic are many factors, including many people and things; different users have different angles, opinions, and positions on trending topics.

[0003] Social networks are teeming with diverse viewpoints and stances, which play a crucial role in identifying fake news and rumors, predicting political trends, and influencing legal amendments. Therefore, analyzing user stances on social networks is vital for maintaining social stability and establishing accurate public opinion. Stance detection, given a specific target entity (person or event) within a given timeframe, assesses users' stance by evaluating their opinions and attitudes towards that entity. Stance detection primarily categorizes users as supportive, opposed, or neutral. Furthermore, with the rise of deep learning, employing its features for text analysis has become a trend.

[0004] Early research on stance detection can be broadly categorized into machine learning-based methods and deep learning-based methods. However, due to the greater complexity and information density of online social networks, stance detection methods specifically designed for online social networks have emerged. For example, many existing research studies have focused on stance detection methods based on machine learning and deep learning.

[0005] Traditional research on user stance detection primarily focuses on analyzing short texts posted by users on social networks. However, this approach can also be considered opinion detection, failing to distinguish between user opinions and stances. While some methods address user stance, they still don't differentiate between opinions and stances. Furthermore, previous stance detection tasks rarely considered external common sense and emotional information, because common sense is perceived as negligible in human cognition, but computers cannot incorporate this information. Summary of the Invention

[0006] This invention addresses the above problems by incorporating vectorized external common sense information to extract sentiment information. Furthermore, to better integrate external common sense knowledge into the neural network, a complementary gate mechanism is added to the Bi-LSTM. Finally, a topic-driven module is designed to acquire user neighbor information, which is then integrated using an LSTM. Combined with an attention mechanism, a user stance detection method based on external common sense knowledge and deep learning is proposed, and a detailed implementation flow is provided.

[0007] The specific technical solution is as follows:

[0008] A user stance detection method based on external common sense knowledge and deep learning includes the following steps:

[0009] Step 1: Define user stance based on tweets posted by users on social networks. User stance refers to the stance a user holds towards a specific target entity within a given time period, by judging the views and attitudes expressed by the user regarding the target event during that time period. The result of user stance is usually one of the following three labels.

[0010] Support: Identify users' supportive and positive attitudes toward the target entity through their historical tweets.

[0011] Neutrality: Identify users' neutral attitudes toward the target entity through their historical tweets.

[0012] Opposition: Identify users' opposing and negative attitudes toward the target entity through their historical tweets.

[0013] Step 2: Social network user data preprocessing, specifically as follows: (1) Delete any characters not within the range of English letters and redundant symbols; (2) Standardize the sequence of repeating characters; (3) Delete all stop words and function words. And set the dimension of short text word embedding to 300 dimensions for training tweet data.

[0014] Step 3: Extract sentiment information using external common sense knowledge. To help filter information from the previous time step to the next, common sense knowledge is used to supplement the original text information. This is achieved by extending the Bi-LSTM and incorporating external common sense into the Bi-LSTM, represented by CK-BiLSTM. At each time step t, it is assumed that each word in the input text can trigger sentiment knowledge and map it to d. c In 3D space, let O represent the number of sentiment knowledge points obtained from the text, expressed by the formula [β]. t，1 ,β t，2 , ..., β t，m , ..., β t，o ], and embed it into the network using an average vector, as shown in Equation (1):

[0015]

[0016] The CK-BiLSTM model with emotional knowledge is represented by formulas (2)-(8):

[0017] f t =σ(W f [w t ,h t-1 ,β t ]+b f (2)

[0018] i t =σ(W i [w t ,h t-1 ,β t ]+b i (3)

[0019]

[0020]

[0021] o t =σ(W o [w t ,h t-1 ,β t ]+b o (6)

[0022]

[0023]

[0024] Among them, f t Represents the forget gate, i t Indicates the input gate, C t Indicates cell state, o tIndicates the output gate. W f W t W C and W o It is called the weight matrix. t and h t These represent the input and hidden states at time t, respectively. t-1 This represents the hidden state at time t-1. f b i b C b o , These represent the bias values ​​for different gates. σ and tanh represent the sigmoid function, and the symbol * represents the Hadamard product. In addition, CK-BiLSTM has an important supplementary activation vector, namely... That is, formula (7) is a Bi-LSTM output gate extended with common sense knowledge. This gate is different from the ordinary gate in Bi-LSTM. t Complementary states are sent to the hidden states, and an emotion is added just to the output gate. The CK-BiLSTM outputs a series of hidden states {h1, ..., h2}. m They are fed into an average pooling layer to obtain the tweet representation c. i .

[0025] Step 4: Topic Information Extraction. Assume x represents a document containing several words, z represents the set of topics in a document, θ represents the set of topic distributions for all topics in the entire corpus, τ represents the set of word distributions for all topics, and γ represents the word distribution for topic z. Then, the LDA model can be generally represented as a joint conditional probability distribution, as shown in the following formula:

[0026] p(x,z,θ,τ|α,β)=p(τ|β)p(θ|α)p(z|θ)p(x|γ)(9)

[0027] Where α represents the model's hyperparameters, referring to the prior distribution preference of the topic distribution. β represents the model's hyperparameters, referring to the prior distribution preference of the word distribution.

[0028] The user stance detection text consists of multiple short comment texts containing various latent topics. The latent topics in user i's historical tweets are denoted as z. An LDA topic model is used to construct the feature representation of the topics, and Gibbs sampling is used for parameter estimation. Then, tweet t... i，n Corresponding topic distribution The estimate is:

[0029]

[0030] Where M represents the counting matrix of the topics corresponding to the documents, Document t represents i，n The total number of times a certain number of words are assigned to topic j, d t The parameter α represents the number of topics in the document, and α represents the prior distribution characteristics of the topics. The number of topics in a tweet is mainly determined by the magnitude of α; a larger α value indicates a larger number of topics in the tweet, and vice versa. The topic probability vector for user i can be obtained through parameter estimation. The resulting topic vector is represented as z.

[0031] Step 5: Extracting text information from neighbors.

[0032] The main method for obtaining user neighbor information is to match it with the topic information already obtained in the previous step. Simultaneously, the model acquires neighbor information in chronological order. First, the hidden vectors of the neighbor information are obtained through CK-BiLSTM, and these vectors are merged and input into the LSTM model. Finally, the LSTM output is obtained by weighting the acquired topic information and neighbor information, as shown in the following formula:

[0033]

[0034]

[0035] in Indicates user's neighbor's tweet N i，l The output hidden vector state, z i To indicate relevant topic information, h t W represents the tweet posted by the user at time step t. h and W z This represents the weight matrix.

[0036] Step 6: The attention mechanism of user history information. When users post tweets, they not only pay attention to topics of interest, but also to the content posted by their neighbors.

[0037] First, for each tweet, its final representation is generated by combining the user's historical tweets with neighbor information obtained from the corresponding topic in the user's tweets.

[0038]

[0039] In this formula, α1 captures the importance of neighbor information obtained under topic-driven conditions, while α2 captures the importance of historical tweets. If α1 < α2, the formula can be interpreted as a model that completely relies on the user's historical tweet information.

[0040] The combined representation is then passed to the user attention layer, whose output t i，n It is {g t，1 g t，1 ,…,g t，i The normalized weighted sum of}.

[0041]

[0042]

[0043] u t,i =tanh(W u g t,i +b) (16)

[0044] In formula (15), β n (n > 0) represents the nth neighbor's tweet that the user follows. In other words, it essentially measures the degree of influence from the nth neighbor's tweet. W and W u Represents the weight matrix, u in formula (16) t，i The function represents the smoothing factor computed from the fully connected layer. Finally, it is fed into the softmax layer for classification.

[0045] y i,n =softmax(t) i,n W y (17)

[0046] Among them W y This represents the weight matrix of the fully connected layer. The output y of the softmax layer... i，n This represents the probability distribution of the end-user's stance category.

[0047] The beneficial effects of this invention are as follows:

[0048] (1) Based on the user's active status in social networks and the use of common sense knowledge, this invention discloses an improved version of the ordinary Bi-LSTM, which sets up a complementary gate mechanism to better obtain the emotional information vector extracted from external common sense knowledge.

[0049] (2) Based on external common sense knowledge, this invention discloses a user stance detection method based on external common sense knowledge and deep learning. Different base classifiers are trained using different emotional features, and a meta-classifier with a global view is obtained using a local fusion strategy. Subsequently, in the global fusion stage, the outputs of the meta-classifier are integrated through an accuracy-based weighted voting scheme to identify emotional roles in social networks. Compared with traditional role recognition methods, this method is more effective in identifying emotional roles. Attached Figure Description

[0050] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0051] The present invention will be further illustrated below with specific examples. These examples are two real social networks, BREXIT and ELECTION, collected from the Internet. Their specific information is shown in Table 1.

[0052] Table 1. BREXIT and ELECTION Network Data Statistics

[0053] Social Networks Number of users Number of tweets Supported quantity Neutral quantity Number of Oppositions BREXIT 38，335 363，961 115，012 142，309 106，640 ELECTION 108，689 452，128 335，479 24，215 92，234

[0054] The process is as follows Figure 1 As shown:

[0055] Step 1: Preprocess the historical tweets obtained from the internet. For example, for the tweet "Hello! I'm Xiaoming...", first remove "ah" and punctuation marks ", ", ", and ", and set the user's tweet sequence to 3.

[0056] Step 2: Obtain sentiment vectors from a common-sense knowledge base. Using the SenticNet knowledge base published by Cambri et al., as shown in Table 2, given a word, the probability of related attributes can be searched using the following list. This knowledge base contains 50,000 common-sense knowledge items and rich sentiment attributes. It not only provides concept-level representations but also semantic connections and sentiment representations. For example, given the fixed word "cake," when people see the word "cake," based on their existing knowledge and common sense, they can deduce that "cake needs to be baked," "cake is edible," "cake is a dessert," "cake needs to be made in an oven," and "cake can solve hunger problems," etc. Furthermore, people are usually happy, joyful, and in a good mood when eating cake. When people use baking to make cake, they usually express anticipation, hoping for the cake to be successfully made. Therefore, when encountering the word "cake," based on people's common-sense knowledge, the word "cake" can usually be associated with the emotions of "happiness" and "anticipation." Then, a series of hidden states are output through CK-BiLSTM and passed to a pooling layer to obtain the tweet representation c. i .

[0057] Table 2 Examples of new knowledge concepts inferred from SenticNet

[0058]

[0059] Step 3: Extracting topic information.

[0060] To obtain the topic information in the tweets, this invention mainly uses the LDA method to extract the topic information. The LDA topic model is used to calculate the topic distribution of each user's topic-level attribute text, and the parameters are estimated by Gibbs sampling. Finally, the topic probability vector of each document can be obtained by calculating formula (10).

[0061] Step 4: Extracting text information from neighbors.

[0062] Previous related patents and research have shown that neighbor text information has a huge impact on some classification problems. Therefore, this invention takes neighbor information into account. In this invention, neighbor information is obtained by using the topic information obtained in step 3, and neighbor information is obtained by formulas (11) and (12).

[0063] Step 5: Attention mechanism for user history information.

[0064] For user stance information, this invention mainly consists of two parts: historical information and neighbor information obtained through topic information. The different degrees of influence of the two parts are obtained through formula (13). Finally, the stance is classified and detected through formula (17).

[0065] This example uses accuracy (Acc), recall (R), and micro F1 score (M-F1), along with their average values, to evaluate the performance of the proposed method. The specific definitions are as follows:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] Where TP represents a correctly identified positive sample, TN represents a correctly identified negative sample, FP represents a incorrectly identified positive sample, and FN represents a incorrectly identified negative sample.

[0073] In the experiment, to illustrate the superiority of the present invention, we selected classic ensemble learning algorithms SVM-ngram, NB, MITRE (RNN), Pkudblab (CNN), TAAT, and Aff-Feature for comparison. The experimental results are shown in Tables 3 and 4.

[0074] Table 3 shows the experimental results on the BREXIT network data.

[0075]

[0076] Table 4 shows the experimental results on the ELECTION network data.

[0077]

[0078] Table 5. Results analysis of Avg_MF1, Avg_Acc, and Avg_R on the two datasets.

[0079]

[0080] Experimental results show that the present invention has significant advantages over the classical algorithm.

Claims

1. A user stance detection method based on external common sense knowledge and deep learning, characterized in that, The method comprises the following steps: Step 1: defining user standpoints according to tweets sent by users in a social network; Step 2: preprocessing social network user data; Step 3: extracting sentiment information using external common sense knowledge; in order to help filter information from the previous time step to the next time step, the original text information is supplemented with common sense knowledge; By extending Bi-LSTM and fusing external common sense into Bi-LSTM, it is denoted as CK-BiLSTM; Step 4: topic information extraction; assuming that x represents a document and the document contains a plurality of words, z represents a topic set in the document, theta represents a topic distribution set of all topics in the entire corpus, tau represents a word distribution set in all topics, and gamma represents a word distribution of the topic z, the LDA model is generally denoted as a joint conditional probability distribution, as shown in the following formula: p(x,z,theta,tau|alpha,beta)=p(tau|beta)p(theta|alpha)p(z|theta)p(x|gamma) (9) Wherein, alpha represents a model hyperparameter, which refers to the prior distribution preference of the topic distribution; beta represents a model hyperparameter, which refers to the prior distribution preference of the word distribution; The user stance detection text consists of multiple comment short texts and contains multiple potential topics; the potential topics in the historical tweets sent by user i are set as z, the LDA topic model is used to construct the feature representation of the topic, and the parameter estimation is carried out through Gibbs sampling; then the tweet t i,n The estimation of the corresponding topic distribution is: Wherein, M represents the count matrix of the document corresponding theme, The total number of documents t i,n The total number of several words assigned to theme j in document t t The number of themes in the document is represented by α, which represents the prior distribution characteristics of the theme distribution, and the number of themes in the tweet is mainly identified by judging the size of α. The larger the value of α, the more the number of themes in the tweet, and vice versa. The smaller the value of α, the fewer the number of themes in the tweet. The theme probability vector of user i is obtained through parameter estimation The obtained theme vector is represented as z; Step 5: neighbor text information extraction; The main method for obtaining user neighbor information is to match user neighbor information through the topic information obtained in the previous step; at the same time, when obtaining user neighbor information, the model obtains information in time sequence; Step 6: attention mechanism of user historical information; when users publish tweets, they pay more attention to the content published by user neighbors while focusing on the topics of interest.

2. The user standpoint detection method based on external common sense knowledge and deep learning according to claim 1, wherein In step 1, the user standpoint refers to, within a certain time period, given a specific target entity, the standpoint of the user held for the target entity is evaluated by judging the viewpoint and attitude of the user published in the time period for the target event; the result of the user standpoint includes the following three labels: Support: the user's tweet is identified to hold a supporting and positive attitude towards the target entity through the user's historical tweets; Neutral: the user's tweet is identified to hold a neutral attitude towards the target entity through the user's historical tweets; Oppose: the user's tweet is identified to hold an opposing and negative attitude towards the target entity through the user's historical tweets. 3.The user stance detection method based on external common sense knowledge and deep learning according to claim 1, characterized in that, The specific operation of step 2 is as follows: (1) deleting any characters not within the range of English letters and redundant symbols; (2) standardizing repeated character sequences; (3) deleting all stop words and virtual words; and setting the dimension of short text word embedding to 300 dimensions for training tweet data. 4.The user stance detection method based on external common sense knowledge and deep learning according to claim 1, characterized in that, Step 3 is specifically: by extending Bi-LSTM and fusing external common sense into Bi-LSTM, it is denoted as CK-BiLSTM; in each time step t, it is assumed that each word in the input text can trigger sentiment knowledge and map it to d c dimensional space; o is used to represent the number of sentiment knowledge obtained in the text, and is expressed by the formula: t，1 β t，2 β t，m β t，O ], and an average vector is used to embed it into the network, as shown in formula (1): (1) The CK-BiLSTM model with sentiment knowledge is represented by formulas (2)-(8): f t =σ(W f [w t ,h t-1 ,b t ]+b f ) (2) i t = σ(W i , h t , β t-1 , a t + b i ) (3) where f t represents the forget gate, i t represents the input gate, C t represents the cell state, o t represents the output gate; W f , W i , W C and W o are called weight matrices; w t and h t represent the input and hidden state at time t, respectively; h t-1 represents the hidden state at time t-1; b f , b i , b c , b o , represent the bias values of different gates; σ and tanh represent the sigmoid function, and the symbol * represents the Hadamard product; in addition, there is an important information supplement activation vector in CK-BiLSTM, that is, that is, formula (7), is a Bi-LSTM output gate extended with common sense knowledge, which is complementary to the ordinary gate o t in Bi-LSTM and sends it to the hidden state, adding emotional knowledge and a supplementary output gate; CK-BiLSTM outputs a series of hidden states {h1, …, h m}, which are transmitted to the average pooling layer to obtain the tweet representation c i . 5.The user stance detection method based on external common sense knowledge and deep learning according to claim 1, characterized in that, Step 5 specifically comprises the following steps: first, the hidden vectors of the neighbor information are obtained through CK-BiLSTM, and they are merged together and input into the LSTM model; finally, the output of the LSTM is weighted by the obtained topic information and neighbor information, and the specific formula is as follows: wherein representing user neighbor tweets N i,l output hidden vector state, z i representing relevant topic information, h t representing tweets posted by users at time step t, W h , and W z representing a weight matrix. 6.The user stance detection method based on external common sense knowledge and deep learning according to claim 1, characterized in that, Step 6 specifically comprises the following steps: First, for each tweet, its final representation is generated by combining the user's historical tweets with the neighbor information obtained from the corresponding topic-driven under the user's tweets; where a1 is used to represent the importance of capturing the neighbor information obtained under the topic-driven, and a2 is used to represent the importance of capturing the historical tweets; if a1 < a2, the formula is understood as a model that completely depends on the user's historical tweet information; The combined representation is then passed to the user attention layer, whose output t i,n is a normalized weighted sum of {g t，0 , g t，1 ,..., g t，i} u t,i = tanh(W u g t,i + b) (16) wherein β n represents the nth neighbor tweet information that the user pays attention to, wherein n > 0; measures the influence degree from the nth neighbor tweet; W and W u represents a weight matrix, wherein u t,i The function represents a smoothing factor calculated from the full connection layer; finally, it is input into the softmax layer for classification; y i,n = softmax(t i,n W y ) (17) where w y represents the weight matrix of the fully connected layer; the output y i,n represents the probability distribution of the final user stance category.

Images