Method for detecting climate change stance in reviews based on hybrid attention and label propagation

Abstract

The application discloses a comment climate change position detection method based on mixed attention and label diffusion, which performs topic modeling on climate change extreme weather event comment data, and summarizes and classifies to obtain high-frequency words of each topic, construct a climate change believer and denier label marking word library, and create a climate change position detection data set by using a label diffusion technology; meanwhile, a climate change position detection model is built, a mixed attention mechanism and a deep learning algorithm are combined to realize climate change position detection on media user comment data, and a position result decision driving factor is visually explained. The method solves the problems of shortage of fine-grained position detection data set in the field of climate change and information feature strengthening extraction, effectively improves the efficiency of position detection work in the field, further understands public opinion emotions about climate change, and is favorable to the formulation and popularization of climate change coping strategies.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and natural language processing, and simultaneously relates to climate science and data analysis, and specifically relates to a comment climate change stance detection method based on mixed attention and label diffusion. BACKGROUND

[0002] Climate change is one of the most serious challenges in the 21st century, and sustained climate change has a wide range of negative impacts on the global economy, social life and natural environment, calling on countries to take more rapid and forceful action to address this global challenge. Although various mitigation measures can be taken to address the intensity and impact of climate change, the actual implementation is closely related to the public's life, and whether these measures can be incorporated into public policy depends on the public's attitude towards climate change. In this context, social media platforms, as research media, not only serve as an important channel for the public to express opinions and share information, but also provide a wealth of real-time data reflecting public attitudes and emotions towards climate change. In order to better develop climate change response strategies, it is absolutely important to understand public opinion and sentiment towards climate change.

[0003] Currently, sentiment analysis techniques are often used to explore public attitudes and information extraction on social media platforms regarding climate change or related events. Based on sentiment analysis, users are divided into three camps: supporters (positive), deniers (negative), and neutral. Ray et al. By comparing two neural networks, CNN and RNN, to find the emotional target, the attitude of media users towards climate change is understood, but the study mainly relies on emotional polarity as the basis for final attitude classification, and does not achieve fine-grained classification to further explore the true stance of users towards climate change; Koenecke et al. For climate change stance detection of Twitter users, a compensation method is proposed to expand the limited artificial stance labeled data, and whether the public stance on climate change will change with natural disasters is explored. Although this method achieves user climate change stance analysis, the feature extraction of the text and the decision factors of the predicted label are not strengthened and explained, which limits the in-depth analysis of media user attitudes and the expansibility of the research.

[0004] Emotional polarity has a certain auxiliary role in the study of media users' attitudes towards climate change, but it cannot represent the true user stance. The creation and expansion of fine-grained stance detection datasets is one of the main limitations of exploring media user climate change stance detection. In addition, media platforms have rich discussions about climate change, covering corporate reports, news reports, and public life, among other aspects. How to strengthen the text feature information is a problem that needs to be considered. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art and provide a comment climate change stance detection method based on mixed attention and label diffusion. This method can solve the problems of lack of fine-grained stance detection data set and information feature extraction enhancement in the field of climate change by using label diffusion technology and mixed attention layer, and further understand the public opinion emotion on climate change, so as to better formulate and promote climate change response strategies.

[0006] The technical solution to achieve the purpose of the present application is:

[0007] The comment climate change stance detection method based on mixed attention and label diffusion comprises the following steps:

[0008] 1) Climate change extreme weather event sub-data set creation: For the existing climate change related comment data set on the Reddit platform, data preprocessing is performed, and different keywords are used to create five extreme weather event sub-data sets of hurricane, flood, wildfire, drought, cold / hot wave;

[0009] 2) Label marking library generation: a text-topic vector is generated for each comment text in each extreme weather event sub-data set created in step 1), and the topics of the five extreme weather event sub-data sets are summarized, classified into climate change supporter and denier stance topics, and the high-frequency words of the same stance topic are collected as the supporter and denier label marking library;

[0010] 3) Label diffusion: 63000 climate change related comment texts are randomly selected from the existing climate change related comment data set on the Reddit platform, and after data preprocessing, the label marking library generated in step 2) is used to mark each comment text to build a climate change stance detection data set;

[0011] 4) Building a climate change stance detection model: the stance detection model is composed of an embedding layer, an information processing layer, a mixed attention layer and an output layer, the comment texts in the climate change stance detection data set built in step 3) are divided into training set, validation set and test set according to the proportion as the input of the stance detection model, the comment texts in the training set and the validation set are used to train the stance detection model, and the trained stance detection model is used to obtain the final climate change stance detection result of the comment texts in the test set;

[0012] 5) Media user climate change stance result analysis: the climate change stance prediction results of each comment text in the test set generated in step 4) are visualized for decision weight, and the stance prediction results are compared with the emotion polarity of the corresponding text marked by the sentiment dictionary, and the fine-grained stance detection effect is viewed through the comparison results.

[0013] The text-topic vector generated in the step 2) is specifically:

[0014] 2.1) Calculate the word weight of each comment text in each extreme weather event sub-data set using the TF-IDF algorithm, which is used as the input of the LDA model to generate the text-topic vector of each comment text, and use pyLDAvis to visualize the modeling effect of different numbers of topics, and select the best number of topics for each extreme weather event sub-data set.

[0015] The embedding layer, information processing layer and hybrid attention layer in the step 4) are specifically:

[0016] 4.1) Use the BERT model as the embedding layer model to obtain the semantic vector of the comment text; use the BiLSTM model as the model of the information processing layer to further integrate the overall information of the comment text and generate a new text representation vector;

[0017] 4.2) In the hybrid attention layer, the representation vector of the text is linearly transformed, and the text features captured in the training process are strengthened by combining the self-attention and label attention hybrid attention mechanism, and the text information related to the predicted label is calculated to understand the decision factors of the predicted label.

[0018] The step 5) is specifically:

[0019] 5.1) Use the attention weight matrix of the hybrid attention layer to visualize the decision weight of the comment text, and the deeper the color of the text word, the greater the influence on the final decision label;

[0020] 5.2) Use the VADER dictionary to mark the emotional polarity of the comment text.

[0021] The beneficial effects of the technical solution are:

[0022] The technical solution makes full use of the word weight, semantic and topic information of the comment text, builds a label marking dictionary based on the climate change related extreme weather event comment data, uses label diffusion technology to build a climate change stance detection data set, and simultaneously builds a climate change stance detection model based on a hybrid attention mechanism, realizes stance detection of climate change related comments, helps to identify supporters, deniers and neutral stance media users, and provides visual explanation factors for stance decision.

[0023] The technical solution has the following advantages:

[0024] (1) This technical solution proposes a climate change stance detection dataset, which conducts topic modeling on data related to five extreme weather events, including droughts, floods, and hurricanes, and summarizes and classifies them, constructs a climate change stance detection label tag vocabulary, and realizes the construction of the climate change stance detection dataset through label diffusion technology. Compared with the high-cost traditional manual labeling or sentiment dictionary labeling limited by sentiment words, it further improves work efficiency, meets the fine-grained classification of public opinion sentiment related to climate change, solves the scale limitation problem, and makes up for the shortage of data in climate change stance detection.

[0025] (2) This technical solution proposes a climate change stance detection model. In the face of the rich expressions of media users' comments, it uses a hybrid attention mechanism combined with a deep learning algorithm to enhance the text information features and capture the text information features that are strongly correlated with stance labels. Based on the sentiment analysis of the comment text, it further classifies and performs climate change stance detection to explore the real attitudes of media users related to climate change.

[0026] (3) This technical solution is based on the attention weight matrix in the hybrid attention mechanism to explain the stance decision factors of media texts. Compared with the traditional climate change sentiment analysis model, it provides more expansion possibilities and is conducive to the combination of downstream tasks and further analysis of public sentiment.

[0027] This method solves the problems of the shortage of fine-grained stance detection datasets and enhanced information feature extraction in the field of climate change through label diffusion technology and hybrid attention layers, further understanding the public's public opinion on climate change, and thus better formulating and promoting climate change response strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is an architectural diagram of an embodiment method;

[0029] Figure 2 A structural diagram of the climate change stance detection model designed for the embodiment;

[0030] Figure 3 This is an example diagram of detecting user comments on climate change stance in an embodiment. DETAILED DESCRIPTION

[0031] The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the invention is not limited thereto.

[0032] Example:

[0033] Reference Figure 1 , a method for detecting climate change stance in comments based on hybrid attention and label diffusion, including the following steps:

[0034] 1) Extreme weather event sub-dataset creation: Based on the existing climate change related comment dataset on Reddit platform, data preprocessing was performed, and different keywords were used to create extreme weather event sub-datasets for hurricanes, floods, wildfires, droughts, and cold / hot waves;

[0035] The existing climate change related comment dataset on Reddit platform was subjected to natural language processing for each line of text, including word segmentation, lowercasing, root restoration, removal of punctuation and stop words preprocessing; and 15 high-frequency related keywords for hurricanes, wildfires, droughts, cold / hot waves were manually evaluated for text extraction. Table 1 is a list of the top 5 keywords for each extreme weather event;

[0036] Table 1

[0037] ;

[0038] 2) Label marking library generation: For each comment text in each extreme weather event sub-dataset created in step 1), a text-topic vector was generated, and the topics were summarized for the five extreme weather event sub-datasets, classified as climate change supporters and deniers stance topics, and the high-frequency words of the same stance topic were collected as the supporter and denier label marking library;

[0039] 2.1) For all comment texts in each extreme weather event sub-dataset, a dictionary text mapping and corpus were created, all words in the comment text were mapped to a unique integer ID, and words with less than 15 occurrences and more than 50% frequency in all comment texts were removed; TF-IDF algorithm was used to weight the corpus, reducing the word weight of frequently occurring but not very helpful for distinguishing documents, and increasing the word weight of documents with text distinguishing degree; The dictionary text mapping and weighted corpus were input into the LDA model, and the entire corpus was traversed 100 times. Due to the differences in text quantity and complexity between each extreme weather event sub-dataset, the best number of topics for each extreme weather event data was determined by modeling 4 to 8 topics for each extreme weather event sub-dataset, and the best number of topics for each extreme weather event sub-dataset was determined using pyLDAvis;

[0040] 2.2) Summarize and classify the topics collected from all extreme weather event sub-datasets, and divide them into six categories: energy consumption, political factors, human life, seasonal disasters, human factors and derivative creative works (such as films, books and games, etc.), and each category is divided into supporter or denier according to the climate change stance of its high-frequency words; that is, the supporter stance topic includes recognizing the impact of human factors on climate, taking measures to reduce climate change, and supporting the concept of climate change, while the denier stance topic believes that the main cause of disaster is not climate change (human, political and seasonal change factors) and denies the concept of climate change. The topic classification of each extreme weather event sub-dataset is shown in Table 2:

[0041] Table 2

[0042] ;

[0043] 2.3) Collect high-frequency words of supporter and denier topics, for each topic, collect the top 30 keywords, delete repeated words, and respectively summarize the supporter stance label tag library and the denier stance label tag library;

[0044] 3) Label diffusion: randomly select 63000 climate change related comment texts from the existing climate change related comment data set on Reddit platform, label them after data preprocessing and label tag library generated in step 2), each comment text has an initial score of 0, the score of the word appearing in the supporter label word library is represented as +1, and the score of the word appearing in the denier label word library is represented as -1. The total score is the final result, if greater than 0, mark the comment user as a supporter, less than 0 as a denier, and equal to 0 as neutral, and build a climate change stance detection data set;

[0045] 4) Build a climate change stance detection model: the stance detection model is composed of embedding layer, information processing layer, mixed attention layer and output layer. The comment texts in the climate change stance detection data set built in step 3) are divided into training set, validation set and test set according to the ratio of 6:2:2 as the input of the stance detection model. The comment texts in the training set and the validation set are used to train the stance detection model, and the trained stance detection model is used to obtain the final climate change stance detection result of the comment texts in the test set;

[0046] 4.1) As shown in Figure 2 , the input text is a sequence of length T, W j =(w1,w2,...,w T ), the BERT model of the embedding layer will encode each word w j to generate a 768-dimensional word vector v j =(v j1 ,v j2,...,v j768 ), and further integrate the sequence information through the BiLSTM model with a hidden unit dimension of N, and output the semantic vector H j =(h1,h2,...,h T ), where h j ∈R 2N ;

[0047] 4.2) In the mixed attention layer, the semantic vector H j Perform linear processing (Formula 1), and then enhance the text features through the self-attention mechanism (Formulas 2 and 3), where W1 and W2 are linear transformation parameters, α ij is the normalized coefficient of the attention weight of the jth word in the i-th sequence text, k i ∈R 2N is the self-attention weight vector of the i-th text sequence;

[0048] (1)

[0049] (2)

[0050] (3)

[0051] After obtaining the self-attention weighted vector of the input sequence, the most relevant text information is calculated for each stance label (Formula 4) to understand the driving factors of the prediction results, and a sequence vector weighted by the mixed attention weight is generated (Formula 5), ​​where β iq is the normalization coefficient, W q ∈R 2N is the attention weight parameter, different labels W q Different; m q ∈R 2N is the mixed attention weighted output of the qth label of the label attention mechanism;

[0052] (4)

[0053] (5);

[0054] 4.3) The final sequence vector is passed to the output layer. Based on the previous attention weighting and linear processing results, the output layer generates probabilities for each category and selects the category with the highest probability as the final climate change stance classification result for the sequence, classifying the input text as either supporter, denier, or neutral.

[0055] 5) Media user climate change stance result analysis: On the prediction results of the climate change stance of each comment text in the test set generated in step 4), the decision weight visualization is performed, and the stance prediction results are compared with the emotion polarity of the corresponding text marked by the sentiment dictionary. Through the comparison results, the fine-grained stance detection effect is viewed, and the effectiveness of this example is further verified.

[0056] 5.1) Use the attention weight matrix of the mixed attention layer to perform decision weight visualization on the comment text. The deeper the color of the text word in the visualization result, the greater the influence on the final decision label of the comment text. As shown in Figure 3 , the text prediction result of the figure is the stance of the supporter, that is, the speaker acknowledges the existence of the climate change stance. According to the word weight heat map, it can be seen that the author thinks that cities are the main factors driving climate change in the past, and has a skeptical attitude towards people who think that climate change can be solved by scientists under the existing technology mitigation;

[0057] 5.2) Use the VADER dictionary to mark the emotion polarity of the Figure 3 comment text, and the marking result is negative. In VADER, the appearance of negative words such as 'don't' and 'isn't' in the text is an important reason for judging negative emotions. In addition, negative words such as 'problem' are also one of the factors of negative scoring, which indirectly proves that the climate change stance detection model in this example is more conducive to exploring the real attitude of the comment user than the traditional emotion dictionary marking for the supporter's stance comment text with negative emotions.

[0058] References:

[0059] [1] Ray, Sujana, and AM Senthil Kumar. "Prediction and Analysis of Sentiments of Reddit Users towards the Climate Change Crisis." 2023 International Conference on Networking and Communications (ICNWC). IEEE, 2023.

[0060] [2] Koenecke, Allison, and Jordi Feliu-Faba. "Learning twitter user sentiments on climate change with limited labeled data." arXiv preprint arXiv:1904.07342 (2019).

Claims

1. A method for detecting the stance on climate change in reviews based on hybrid attention and label propagation, characterized in that, Comprise the following steps: 1) Extreme weather event sub-data set creation: For the existing climate change related comment data set on Reddit platform, after data preprocessing, different keywords are used to create extreme weather event sub-data sets for hurricanes, floods, wildfires, droughts, cold / hot waves; 2) Label marking library generation: For each comment text in each extreme weather event sub-data set created in step 1), a text-topic vector is generated, and the topics of the five extreme weather event sub-data sets are summarized, and the summarized topics are classified into climate change supporter and denier stand topics, wherein, for the five extreme weather event sub-data sets, 4 to 8 topics are modeled, and the best number of topics for each extreme weather event sub-data set is determined by pyLDAvis; Then summarize and classify the topics collected from all extreme weather event sub-data sets, and distinguish the supporter or denier stand according to the climate change stand tendency of the high-frequency words of each topic, and collect the high-frequency words of the same stand topic as the supporter and denier label marking library; 3) Label diffusion: 63000 comment texts are randomly selected from the existing climate change related comment data set on Reddit platform, and after data preprocessing, each comment text is labeled by the label marking library generated in step 2), each comment text has an initial score of 0, and the score of the word appearing in the supporter marking word library is represented as +1, and the score of the word appearing in the denier marking word library is represented as -1, and the total score is calculated as the final result, if greater than 0, mark the comment user as a supporter, less than 0 as a denier, and equal to 0 as neutral, to build a climate change stand detection data set; 4) Build a climate change stand detection model: The stand detection model is composed of embedding layer, information processing layer, hybrid attention layer and output layer, the comment texts in the climate change stand detection data set built in step 3) are divided into training set, validation set and test set according to the proportion as the input of the stand detection model, the comment texts in the training set and the validation set are used to train the stand detection model, and the trained stand detection model is used to obtain the final climate change stand detection result of the comment texts in the test set; In the hybrid attention layer, the text representation vector is linearly transformed, the hybrid attention mechanism combining self-attention and label attention is used to strengthen the features of the comment text captured in the training process, and the text information related to the predicted label is calculated to understand the decision factors of the predicted label; 5) Analysis of media user climate change stand results: The climate change stand prediction results of each comment text in the test set generated by step 4) are visualized, and the stand prediction results are compared with the emotion polarity of the corresponding text marked by the emotion dictionary, and the fine-grained stand detection effect is viewed through the comparison results.

2. The method for detecting the stance on climate change of the review based on mixed attention and label propagation according to claim 1, wherein, The text-topic vector generated in step 2) is specifically: 2.1) Adopting The word weights of each review text in each extreme weather event sub-data set are calculated by the algorithm, which generates the text-topic vector for each review text as the input of the LDA model, and the modeling effect of different numbers of topics is visualized by pyLDAvis to select the optimal number of topics for each extreme weather event sub-data set.

3. The method for detecting the stance on climate change of the review based on mixed attention and label propagation according to claim 1, wherein, The embedding layer, information processing layer and hybrid attention layer in step 4) are specifically: 4.1) BERT model is used as the embedding layer model to obtain the semantic vector of the review text; BiLSTM model is used as the model of the information processing layer to further integrate the overall information of the review text and generate a new text representation vector; 4.2) In the mixed attention layer, the text representation vector is linearly transformed, the mixed attention mechanism combining self-attention and label attention is used to strengthen the features of the review text captured in the training process, and the text information related to the predicted label is calculated to understand the decision factors of the predicted label.

4. The method for detecting the stance on climate change of a review based on mixed attention and label propagation according to claim 1, wherein, The step 5) is specifically: 5.1) The attention weight matrix of the mixed attention layer is used to visualize the decision weight of the review text, and the deeper the color of the text word in the visualization result, the greater the influence on the final decision label; 5.2) The VADER dictionary is used to mark the emotional polarity of the review text.

Images