Comment climate change site detection method based on mixed attention and label diffusion

Abstract

The invention discloses a comment climate change site detection method based on mixed attention and label diffusion, and the method comprises the steps: carrying out the topic modeling of climate change extreme weather event comment data, summarizing and classifying, obtaining the high-frequency words of each topic, and constructing a climate change believer and denier label marking word library, creating a climatic change site detection data set by using a label diffusion technology; meanwhile, a climatic change field detection model is built, climatic change field detection of media user comment data is achieved in combination with a mixed attention mechanism and a deep learning algorithm, and field result decision driving factors are visually explained. According to the method, the problems of shortage of fine granularity site detection data sets in the climatic change field and enhanced extraction of information features are solved, the efficiency of site detection work in the field is effectively improved, public opinion moods about climatic change of the public are further understood, and making and popularization of climatic change coping strategies are facilitated.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and natural language processing, and also involves climate science and data analysis. Specifically, it is a method for detecting the stance of climate change in comments based on hybrid attention and label diffusion. Background Art

[0002] Climate change is one of the most serious challenges of the 21st century. The World Economic Forum pointed out in its 2023 report that continued climate change has a wide range of negative impacts on the global economy, social life and natural environment, and called on countries to take more rapid and forceful actions to address this global challenge. Although a variety of mitigation measures can be taken to address the intensity and impact of climate change, the actual implementation is closely related to the lives of the public. Whether these measures can be incorporated into public policies depends on the public's attitude towards climate change. In this context, social media platforms, as research media, are not only an important channel for the public to express their opinions and share information, but also provide a large amount of real-time data, reflecting the society's attitude and emotions towards climate change. In order to better formulate climate change response strategies, it is absolutely important to understand the public's public opinion on climate change.

[0003] Currently, sentiment analysis technology is often used to explore the public's attitudes towards climate change or related events and extract information on social media platforms. Based on sentiment analysis, users are divided into three camps: believers (positive), deniers (negative), and neutral. Ray et al. [1] By comparing two neural networks, CNN and RNN, the emotional targets were found to understand the attitudes of media users on climate change. However, their research mainly relied on emotional polarity as the basis for the final attitude classification, and did not achieve fine-grained classification to further explore the users' true stance on climate change; Koenecke et al. [2] For detecting Twitter users’ stance on climate change, a compensation method is proposed to expand the limited manual stance labeling data, and whether the public’s stance on climate change will change with the occurrence of natural disasters is explored. Although this method realizes the analysis of users’ stance on climate change, the feature extraction of text and the decision factors of predicting labels are not strengthened and explained, which limits the in-depth analysis of media users’ attitudes and the scalability of the research.

[0004] Emotional polarity plays a certain auxiliary role in the study of media users' attitudes towards climate change, but it cannot represent the real user positions. The creation and expansion of fine-grained position detection datasets is one of the main limitations in exploring media users' position detection on climate change. In addition, the discussion on climate change on media platforms has rich expressions, covering corporate reports, news reports, public life and other aspects. How to strengthen text feature information is an issue that needs to be considered. Summary of the invention

[0005] The purpose of the present invention is to provide a method for detecting the stance of climate change in comments based on hybrid attention and label diffusion in view of the shortcomings of the prior art. This method can solve the problem of shortage of fine-grained stance detection datasets in the field of climate change and enhanced extraction of information features through label diffusion technology and hybrid attention layer, further understand the public's public opinion on climate change, and thus better formulate and promote climate change response strategies.

[0006] The technical solution for achieving the purpose of the present invention is:

[0007] The method for detecting climate change stance in comments based on hybrid attention and label diffusion includes the following steps:

[0008] 1) Creation of sub-datasets of extreme weather events of climate change: For the existing climate change-related comment datasets on the Reddit platform, data preprocessing was performed, and extreme weather event sub-datasets were created for the five extreme weather events of hurricanes, floods, wildfires, droughts, and cold / heat waves using different keywords;

[0009] 2) Label tag word library generation: Generate a text-topic vector for each comment text in each extreme weather event sub-dataset created in step 1), summarize the topics of the five extreme weather event sub-datasets, classify the summarized topics into climate change believers and deniers’ stance topics, and collect high-frequency words in the same stance topic as believers and deniers’ label tag library;

[0010] 3) Label diffusion: 63,000 climate change-related comment texts are randomly selected from the existing climate change-related comment data set on the Reddit platform. After data preprocessing, each comment text is labeled using the label tag library generated in step 2) to construct a climate change stance detection dataset;

[0011] 4) Build a climate change stance detection model: The stance detection model consists of an embedding layer, an information processing layer, a mixed attention layer, and an output layer. The comment texts in the climate change stance detection dataset constructed in step 3) are divided into a training set, a validation set, and a test set in proportion as the input of the stance detection model. The stance detection model is trained with the comment texts in the training set and the validation set, and the trained stance detection model is used to obtain the final climate change stance detection results of the comment texts in the test set.

[0012] 5) Analysis of media users’ stance on climate change: The decision weights of the climate change stance prediction results generated by step 4) for each comment text in the test set are visualized. At the same time, the stance prediction results are compared with the emotional polarity of the corresponding text annotated by the sentiment dictionary. The comparison results are used to check the fine-grained stance detection effect.

[0013] The text-topic vector generated in step 2) is specifically:

[0014] 2.1) The TF-IDF algorithm is used to calculate the word weight of each comment text in each extreme weather event sub-dataset. The word weight is used as the input of the LDA model to generate a text-topic vector for each comment text. pyLDAvis is used to visualize the modeling effect of different numbers of topics and select the optimal number of topics for each extreme weather event sub-dataset.

[0015] The embedding layer, information processing layer, and hybrid attention layer in step 4) are specifically:

[0016] 4.1) The BERT model is used as the embedding layer model to obtain the semantic vector of the comment text; the BiLSTM model is used as the information processing layer model to further integrate the overall information of the comment text and generate a new text representation vector;

[0017] 4.2) In the hybrid attention layer, the representation vector of the text is linearly transformed. By combining the hybrid attention mechanism of self-attention and label attention, the text features captured during the training process are enhanced, and the text information related to the predicted label is calculated to understand the decision factors of the predicted label.

[0018] The step 5) is specifically as follows:

[0019] 5.1) Use the attention weight matrix of the hybrid attention layer to visualize the decision weights of the comment text. The darker the text word in the visualization result, the greater its influence on the final decision label;

[0020] 5.2) Use the VADER dictionary to annotate the sentiment polarity of the review text.

[0021] Beneficial effects of this technical solution:

[0022] This technical solution makes full use of the word weights, semantics and topic information of the comment text, builds a label tag vocabulary based on the comment data of extreme weather events related to climate change, uses label diffusion technology to construct a climate change stance detection dataset, and builds a climate change stance detection model based on the hybrid attention mechanism to achieve stance detection of climate change related comments, help identify believers, deniers and neutral media users, and provide visual explanation factors for stance decision-making.

[0023] This technical solution has the following advantages:

[0024] (1) This technical solution proposes a climate change stance detection dataset, which performs topic modeling on data related to five extreme weather events, including droughts, floods, and hurricanes, and summarizes and classifies them to construct a climate change stance detection label tagging vocabulary. The construction of the climate change stance detection dataset is achieved through label diffusion technology. Compared with the costly traditional manual labeling or sentiment dictionary labeling limited by sentiment words, it further improves work efficiency, meets the fine-grained classification of public opinion sentiment related to climate change, solves the scale limitation problem, and makes up for the shortage of data in climate change stance detection.

[0025] (2) This technical solution proposes a climate change stance detection model. In the face of the rich expressions of media users' comments, it uses a hybrid attention mechanism combined with a deep learning algorithm to strengthen the text information features and capture the text information features that are strongly correlated with the stance label. It further classifies the comments based on the sentiment analysis, performs climate change stance detection, and explores the media users' true attitudes towards climate change.

[0026] (3) Based on the attention weight matrix in the hybrid attention mechanism, this technical solution interprets the stance decision factors of media texts. Compared with the traditional climate change sentiment analysis model, it provides more expansion possibilities and is conducive to the combination of downstream tasks and further analysis of public sentiment.

[0027] This method solves the problem of shortage of fine-grained stance detection datasets and enhanced extraction of information features in the field of climate change through label diffusion technology and hybrid attention layer, further understands the public opinion on climate change, and thus better formulates and promotes climate change response strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of an embodiment method;

[0029] Figure 2 A structural diagram of the climate change stance detection model designed for the embodiment;

[0030] Figure 3 This is an example diagram of detecting user comments on climate change stance in an embodiment. DETAILED DESCRIPTION

[0031] The invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, but the invention is not limited thereto.

[0032] Example:

[0033] Reference Figure 1 , a method for detecting climate change stance in comments based on hybrid attention and label diffusion, including the following steps:

[0034] 1) Creation of sub-datasets of extreme weather events of climate change: We preprocessed the existing climate change-related comment datasets on the Reddit platform and used different keywords to create extreme weather event sub-datasets for five extreme weather events: hurricanes, floods, wildfires, droughts, and cold / heat waves;

[0035] The existing climate change-related comment dataset on the Reddit platform is used to perform natural language processing on each line of text in the dataset, including word segmentation, lowercase conversion, root restoration, punctuation removal, and stop word preprocessing. In addition, 15 high-frequency related keywords are manually evaluated for five extreme weather events: hurricanes, wildfires, droughts, and cold / heat waves. Table 1 is a list of the top 5 keywords in terms of frequency of each extreme weather event.

[0036] Table 1

[0037]

[0038]

[0039] 2) Label tag word library generation: Generate a text-topic vector for each comment text in each extreme weather event sub-dataset created in step 1), summarize the topics of the five extreme weather event sub-datasets, classify the summarized topics into climate change believers and deniers’ stance topics, and collect high-frequency words in the same stance topic as believers and deniers’ label tag library;

[0040] 2.1) For all the comment texts in each extreme weather event sub-dataset, create a dictionary text mapping and a corpus. All words in the comment text are mapped to a unique integer ID, and words that appear less than 15 times and appear more than 50% of the time in all comment texts are removed; the TF-IDF algorithm is used to weight the word frequency of the corpus, reducing the weight of words that appear frequently but are not very helpful in distinguishing documents, and increasing the weight of words with textual discrimination; the dictionary text mapping and the weighted corpus are input into the LDA model, and the entire corpus is traversed 100 times. Because the number and complexity of texts vary among the extreme weather event sub-datasets, in order to determine the optimal number of topics for each extreme weather event data, topic modeling with 4 to 8 topics is performed on the five extreme weather event sub-datasets, and pyLDAvis is used to determine the optimal number of topics for each extreme weather event sub-dataset;

[0041] 2.2) The topics collected from all extreme weather event sub-datasets were summarized and classified into six categories: energy consumption, political factors, human life, seasonal disasters, human factors, and derivative cultural creations (such as movies, books, games, etc.). Each category of topics was divided into believers or deniers according to the climate change stance of its high-frequency words; that is, the believer stance theme included acknowledging that human factors have an impact on the climate, taking measures to reduce climate change, and supporting the concept of climate change. The denier stance theme believed that the main cause of the disaster was non-climate change (human politics, seasonal change factors), and denied the concept of climate change. The classification of the topics of each extreme weather event sub-dataset is shown in Table 2:

[0042] Table 2

[0043]

[0044]

[0045] 2.3) Collect high-frequency words of believer and denier topics. For each topic, collect the top 30 keywords with the highest frequency, delete the repeated words, and summarize them into the believer position label tag library and the denier position label tag library respectively;

[0046] 3) Label diffusion: 63,000 climate change related comment texts were randomly selected from the existing climate change related comment data set on the Reddit platform. After data preprocessing, they were labeled using the label tag library generated in step 2). The initial score of each comment text was 0, and the score of the word appearing in the believer tag word library was represented by +1, and the score of the denier tag word library was represented by -1. The total score was calculated as the final result. If it was greater than 0, the comment user was marked as a believer, less than 0, a denier, and equal to 0, a neutral, to construct a climate change stance detection dataset;

[0047] 4) Build a climate change stance detection model: The stance detection model consists of an embedding layer, an information processing layer, a mixed attention layer, and an output layer. The comment texts in the climate change stance detection dataset constructed in step 3) are divided into a training set, a validation set, and a test set in a ratio of 6:2:2 as the input of the stance detection model. The stance detection model is trained with the comment texts in the training set and the validation set, and the trained stance detection model is used to obtain the final climate change stance detection results of the comment texts in the test set.

[0048] 4.1) Figure 2 As shown, the input text is a sequence text W of length T j =(w1,w2,...,w T ), the BERT model of the embedding layer will be for each word w j Encode and generate a 768-dimensional word vector vj =(v j1 ,v j2 ,...,v j768 ), and further integrate the sequence information through the BiLSTM model with a hidden unit dimension of N, and output the semantic vector H j =(h1,h2,...,h T ), where h j ∈R 2N ;

[0049] 4.2) In the mixed attention layer, the semantic vector H j Perform linear processing (Formula 1), and then enhance the text features through the self-attention mechanism (Formulas 2 and 3), where W1 and W2 are linear transformation parameters, α ij is the normalized coefficient of the attention weight of the jth word in the i-th sequence text, k i ∈R 2N is the self-attention weighted vector of the i-th text sequence;

[0050] X=W2tanh(W1H)(1)

[0051]

[0052]

[0053] After obtaining the self-attention weighted vector of the input sequence, the most relevant text information is calculated for each stance label (Formula 4) to understand the driving factors of the prediction results, and a sequence vector weighted by mixed attention weights is generated (Formula 5), ​​where β iq is the normalization coefficient, W q ∈R 2N is the attention weight parameter, different labels W q Different; m q ∈R 2N is the mixed attention weighted output of the qth label of the label attention mechanism;

[0054]

[0055] 4.3) The final sequence vector is passed to the output layer; the output layer generates the probability of each category based on the previous attention weighting and linear processing results, and selects the category with the highest probability as the final climate change stance classification result of the sequence, classifying the input text into climate change believers, deniers, and neutral positions;

[0056] 5) Analysis of media users’ stance on climate change: The decision weights of the climate change stance prediction results generated by step 4) for each comment text in the test set are visualized. At the same time, the stance prediction results are compared with the emotional polarity of the corresponding text annotated by the sentiment dictionary. The fine-grained stance detection effect is checked by comparing the results to further verify the effectiveness of this example.

[0057] 5.1) The decision weights of the comment text are visualized using the attention weight matrix of the hybrid attention layer. The darker the text word in the visualization result, the greater its influence on the final decision label of the comment text; Figure 3 As shown in the figure, the text prediction result is a believer's stance, that is, the speaker acknowledges the existence of climate change stance. According to the word weight heat map, the author believes that cities were the main factor driving climate change in the past, and is skeptical of those who believe that climate change can be solved by scientists under the current technological mitigation conditions.

[0058] 5.2) Using VADER dictionary Figure 3 The comment text was annotated with sentiment polarity, and the result was negative. In sentiment tagging dictionaries such as VADER, the appearance of negative words 'don't' and 'isn't' in the text is an important reason for judging negative sentiment. In addition, negative words such as 'problem' are also one of the factors for negative scoring. This indirectly confirms that the climate change stance detection model in this example is more conducive to exploring the true attitude of the comment user for negative sentiment believer stance comment texts than traditional sentiment dictionary annotations.

[0059] References:

[0060] [1] Ray, Sujana, and AM Senthil Kumar. "Prediction and Analysis of Sentiments of Reddit Users towards the Climate Change Crisis." 2023International Conference on Networking and Communications (ICNWC). IEEE, 2023.

[0061] [2]Koenecke, Allison, and Jordi Feliu-Faba. "Learning twitter usersentiments on climate change with limited labeled data." arXiv preprint arXiv:1904.07342(2019).

Claims

1. A method for detecting climate change stance in comments based on hybrid attention and label diffusion, characterized by: The steps include: 1) Creation of sub-datasets of extreme weather events of climate change: After preprocessing the existing climate change-related comment dataset on the Reddit platform, we used different keywords to create extreme weather event sub-datasets for five extreme weather events: hurricanes, floods, wildfires, droughts, and cold / heat waves; 2) Label tag word library generation: Generate a text-topic vector for each comment text in each extreme weather event sub-dataset created in step 1), summarize the topics of the five extreme weather event sub-datasets, classify the summarized topics into climate change believers and deniers’ stance topics, and collect high-frequency words in the same stance topic as believers and deniers’ label tag library; 3) Label diffusion: 63,000 comments are randomly selected from the existing climate change-related comment dataset on the Reddit platform. After data preprocessing, each comment text is labeled using the label tag library generated in step 2) to construct a climate change stance detection dataset; 4) Build a climate change stance detection model: The stance detection model consists of an embedding layer, an information processing layer, a mixed attention layer, and an output layer. The comment texts in the climate change stance detection dataset constructed in step 3) are divided into a training set, a validation set, and a test set in proportion as the input of the stance detection model. The stance detection model is trained with the comment texts in the training set and the validation set, and the trained stance detection model is used to obtain the final climate change stance detection results of the comment texts in the test set. 5) Analysis of media users’ stance on climate change: The decision weights of the climate change stance prediction results generated by step 4) for each comment text in the test set are visualized. At the same time, the stance prediction results are compared with the emotional polarity of the corresponding text annotated by the sentiment dictionary. The fine-grained stance detection effect can be checked through the comparison results.

2. The method for detecting climate change stance in comments based on hybrid attention and label diffusion according to claim 1 is characterized in that: The text-topic vector generated in step 2) is specifically: 2.1) The TF-IDF algorithm is used to calculate the word weight of each comment text in each extreme weather event sub-dataset. The word weight is used as the input of the LDA model to generate a text-topic vector for each comment text. pyLDAvis is used to visualize the modeling effect of different numbers of topics and select the optimal number of topics for each extreme weather event sub-dataset.

3. The method for detecting climate change stance in comments based on hybrid attention and label diffusion according to claim 1 is characterized in that: The embedding layer, information processing layer, and hybrid attention layer in step 4) are specifically: 4.1) The BERT model is used as the embedding layer model to obtain the semantic vector of the comment text; the BiLSTM model is used as the information processing layer model to further integrate the overall information of the comment text and generate a new text representation vector; 4.2) In the hybrid attention layer, the text representation vector is linearly transformed. The review text features captured during the training process are enhanced through a hybrid attention mechanism that combines self-attention and label attention, and the text information related to the predicted label is calculated to understand the decision factors of the predicted label.

4. The method for detecting climate change stance in comments based on hybrid attention and label diffusion according to claim 1, characterized in that: The step 5) is specifically as follows: 5.1) Use the attention weight matrix of the hybrid attention layer to visualize the decision weights of the comment text. The darker the text word in the visualization result, the greater its influence on the final decision label; 5.2) Use the VADER dictionary to annotate the sentiment polarity of the review text.

Images