A disaster information filtering method and system based on graph attention network

Abstract

The present invention discloses a disaster information filtering method based on a graph attention network, which has the following characteristics: using a graph attention network to understand the correlation between the words in a post and the corresponding information type, and filtering to obtain executable information, including the following steps: step 1, preprocessing the posts in the data set to obtain preprocessed data; step 2, building an information filtering network model; step 3, inputting the preprocessed data into the information filtering network model for training to obtain a trained information filtering network model; step 4, inputting the posts to be classified into the trained information filtering network model to obtain a classification result. Among them, the information filtering network includes a BERT encoder, a graph attention network, and a relationship network. The present invention also discloses a disaster information filtering system based on a graph attention network, including a preprocessing unit and an information filtering unit.

Description

Technical Field

[0001] The present invention belongs to the field of machine learning, and in particular relates to a disaster information filtering method and system based on a graph attention network. Background Art

[0002] Social media has become a crucial medium for sharing information during emergencies. A key difference between social media and traditional news sources is the ability to receive real-time feedback from affected communities. Relief organizations can benefit from this two-way communication channel, both keeping people informed and gaining insights from the updates received by those affected. Therefore, extracting crisis information from posts on social platforms can leverage situational awareness and enable faster responses.

[0003] Most research approaches information extraction from social media as a binary text classification problem. Studies have demonstrated the role of social media as a primary source of information during disasters. Some studies focus on filtering relevant information from posts, while others propose annotation schemes to classify posts into fine-grained labels that take into account people's attitudes, information sources, and decision-making behaviors before, during, and after a disaster. To advance the state of social media crisis monitoring solutions, initiatives have been launched in recent years. One such initiative is Event Stream Tracking, which began at the Text Retrieval Conference in 2018. This project aims to classify disaster-related tweets into multiple information types.

[0004] At the same time, post classification is a challenging natural language processing task for short text. This is because posts lack sufficient contextual information and are inherently noisy, such as misspellings, abbreviations, and emoticons. Furthermore, classification becomes even more challenging in the multi-label context, as a post may belong to one or more labels simultaneously. Early research in text classification separated feature engineering and model training into distinct subtasks. With the advent of end-to-end deep learning methods and attention mechanisms, significant progress has been made in the field of multi-label text classification. Traditional methods can only appropriately extract features from text.

[0005] However, these methods ignore the structural information and relationships between words in the text corpus. Pre-trained language models, such as BERT, have become increasingly popular for text classification tasks. However, because BERT only captures local contextual information, BERT embeddings cannot fully capture global information about the language vocabulary. To address this issue, graph-based methods such as graph convolutional networks and graph attention networks are promising for understanding the global relationships between words in the vocabulary.

[0006] Recent research has leveraged the advantages of combining BERT and graph networks. For an input sentence, BERT embedding is used to capture local contextual information, while graph embedding is used to extract global information about each word in the sentence, which is then concatenated with BERT. The two representations from BERT and graph convolutional networks then interact through a self-attention mechanism to perform classification tasks.

[0007] However, there is currently a lack of effective systems to map relevant posts to more fine-grained labels. Such fine-grained labels are particularly valuable to crisis responders as they can filter key information for rapid disaster response. Summary of the Invention

[0008] The present invention is made to solve the above problems, and its purpose is to provide a disaster information filtering method and system based on graph attention network.

[0009] The present invention provides a disaster information filtering method based on a graph attention network. The method uses the graph attention network to understand the correlation between words in a post and the corresponding information type, filtering to obtain actionable information. The method comprises the following steps: Step 1: preprocessing posts in a dataset to obtain preprocessed data; Step 2: building an information filtering network model; Step 3: inputting the preprocessed data into the information filtering network model for training to obtain a trained information filtering network model; Step 4: inputting the posts to be classified into the trained information filtering network model to obtain classification results. The information filtering network includes a BERT encoder, a graph attention network, and a relational network.

[0010] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein the BERT encoder captures the semantics of the tweet and represents it as a low-dimensional vector.

[0011] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein, in step 2, a learnable distance metric is used to learn the similarity between the post vector and the label vector in a supervised manner.

[0012] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein, in step 2, the information filtering network model learns the TREC-IS dataset and adopts the actionable information definition from the TREC-IS challenge author.

[0013] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein, in step 2, the cumulative alarm value is used to evaluate the performance of the information filtering network model to identify actionable information in tweets.

[0014] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein, in step 2, a meta-learning method is used to learn the mapping between input features and multi-label outputs.

[0015] The disaster information filtering method based on graph attention network provided by the present invention may also have the following features: wherein, in step 2, a supervised learning method is used to train the relational network to learn similarity.

[0016] In the disaster information filtering method based on graph attention network provided by the present invention, it can also have the following characteristics: wherein, step S3 includes the following steps: step S3-1, inputting the preprocessed data into the information filtering network model in sequence and performing one iteration; step S3-2, after the iteration, using the model parameters of the last layer to calculate the loss error respectively, and then backpropagating the loss error to update the model parameters; step S3-3, repeating steps S3-1 to S3-2 until the training completion conditions are met, and obtaining the trained information filtering network model.

[0017] The present invention provides a disaster information filtering system based on a graph attention network. The system uses the graph attention network to understand the correlation between words in a post and the corresponding information type, filtering to obtain executable information. The system includes: a preprocessing unit that preprocesses posts in a dataset to obtain preprocessed data; an information filtering unit that builds an information filtering network model, inputs the preprocessed data into the information filtering network model for training, and obtains a trained information filtering network model. The system then inputs posts to be classified into the trained information filtering network model to obtain classification results. The information filtering network includes a BERT encoder, a graph attention network, and a relational network.

[0018] Functions and effects of the invention

[0019] According to the disaster information filtering method based on graph attention network involved in the present invention, the graph attention network is used to understand the correlation between the words of the post and the corresponding information type, and executable information is filtered, because it includes the following steps: Step 1, preprocessing the posts in the data set to obtain preprocessed data; Step 2, building an information filtering network model; Step 3, inputting the preprocessed data into the information filtering network model for training to obtain a trained information filtering network model; Step 4, inputting the posts to be classified into the trained information filtering network model to obtain classification results. Among them, the information filtering network model includes a BERT encoder, a graph attention network and a relationship network.

[0020] Therefore, the present invention adopts a model that combines the BERT encoder and the audio-graph attention network. This model can integrate the contextual representation of tweets with the correlation between tweets and their tags. This can automatically classify tweets into multi-label information types and filter key information from large amounts of social media data.

[0021] In addition, the disaster information filtering method and system based on graph attention network of the present invention are particularly suitable for use in social platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a disaster information filtering method based on a graph attention network in an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of the structure of an information filtering network in an embodiment of the present invention; and

[0024] Figure 3 2 is a schematic diagram of the structure of the attention network in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate a disaster information filtering method and system based on a graph attention network of the present invention.

[0026] In this embodiment, a disaster information filtering method based on graph attention network is provided.

[0027] The platform implemented in this embodiment has an operating system of Ubuntu 16.04, is implemented using Python 2.7, uses Tensorflow-gpu 1.8.0 version for the neural network framework, CUDA version 9.0, and uses an NVIDIA 1080Ti GPU for the computing acceleration unit.

[0028] The datasets used in this example are two public datasets provided by TREC. The TREC-IS dataset contains approximately 35,000 posts collected during 33 different disasters between 2012 and 2019, including types such as wildfires, earthquakes, hurricanes, explosions, and floods. These posts were labeled with 25 information types by human experts and volunteers. The COVID-19 post dataset contains a collection of posts about the COVID-19 epidemic in different affected areas. In total, the data has 7,590 posts, and like the TREC-IS dataset, it is labeled with one or more of the complete 12 information type labels.

[0029] This example splits each dataset into an 80%-20% ratio, where 80% of the posts are used for training and 20% for testing. During the training phase, 20% of the training data is used to validate the model.

[0030] This example first preprocesses the posts in the dataset, then builds a BERT encoder and graph attention network, trains the information filtering network, evaluates it on a test set, and finally feeds the trained model into the posts to be classified to obtain the classification results. This involves four steps: preprocessing, building the information filtering network model, training the information filtering network model, and feeding the posts to be classified into the trained model to obtain the results.

[0031] Figure 1 4 is a flowchart of the disaster information filtering method based on graph attention network in this embodiment.

[0032] like Figure 1 As shown, the process of the disaster information filtering method based on graph attention network in this embodiment includes the following steps:

[0033] Step S1: preprocess the posts in the data set to obtain preprocessed data.

[0034] In this example, the post content comes from posts in the dataset, and special preprocessing steps are required to capture the semantics of the post. The preprocessing process is as follows:

[0035] First, we used NLTK's TweetTokenize module to tokenize the posts while preserving the text content. Next, we removed stop words, URLs, usernames, and Unicode characters. Furthermore, we removed extra spaces, repeated periods, question marks, and exclamation points. Finally, we used the emoji library to convert emojis to text. Finally, we used the spaCy library to extract named entities from the posts.

[0036] Step S2: Building an information filtering network model.

[0037] In this example, we use the existing deep learning framework TensorFlow to build an information filtering network model. The first component of the information filtering network is the BERT encoder, which converts the input post into a vector representation of its context. The BERT encoder divides the post into multiple tokens and outputs an embedding vector. The BERT encoder captures the semantics of the tweet and represents it as a low-dimensional vector.

[0038] Figure 2 It is a structural diagram of the information filtering network in the embodiment of the present invention.

[0039] like Figure 2 As shown in the figure, BERT encoding specifically includes the following structure:

[0040] The BERT encoder consists of 12 encoding blocks, 768 hidden dimensions, and 12 attention heads. Furthermore, BERT inputs are specially preprocessed. A token is added to the beginning of a post and a token is added after each sentence to indicate sentence boundaries. Each token is partitioned into three embeddings: token, segment, and position. These three embeddings are summed to form a single output vector that captures the meaning of the input post.

[0041] The second component of the information filtering network is the graph attention network. The graph attention network learns the structural information between nodes based on the relative importance of adjacent nodes. The learned label representation is then extracted and concatenated with the post vector as the input of the final component.

[0042] Figure 3 Schematic diagram of the structure of the attention network in the figure in this embodiment.

[0043] like Figure 3 As shown in Figure 2, the Graph Attention Network first constructs a graph from the dataset. Each node can be a word, a named entity, or a label. A feature matrix is ​​used to represent the node, where each element in the matrix is ​​the feature vector of the node.

[0044] First, node representations are initialized using pre-trained embeddings. Next, the relationships between nodes are modeled using an adjacency matrix. The graph attention network consists of two graph attention layers. Each layer takes a node's feature matrix as input and, through an attention operation, learns a new feature matrix based on the importance of each node's neighbors. A common attention mechanism is applied to all nodes. The attention operation is a single-layer feedforward network with the same parameter matrix for each node. Finally, the softmax function is used to normalize the attention scores.

[0045] The third component of the information filtering network is the relational network. The purpose of constructing the relational network is to learn a similarity metric between post vectors and tag vectors using supervised learning. In this embodiment, supervised learning is used to train the relational network to learn similarity.

[0046] Furthermore, a neural network is used as a learnable nonlinear distance function to learn how to match the similarity between the post vector and each tag. The relational network takes the connection matrix output by the BERT encoder as input and outputs a tag vector, using binary cross-entropy as the loss function. Next, a sigmoid function is used at the output layer to calculate the independent likelihood of each tag among all possible tags. Finally, a set of related tags is returned as the final output.

[0047] Step S3: input the pre-processed data into the information filtering network model for training to obtain a trained information filtering network model.

[0048] Hyperparameters were optimized using a grid search method to find the optimal values ​​for best performance. The batch size was set to 128, and the number of training epochs was set to 200. The Adam algorithm was used for optimization, with an initial learning rate of 0.00002. To avoid overfitting, a dropout layer was added, and early stopping was applied during model training.

[0049] The specific implementation process is:

[0050] Step S3-1: input the pre-processed data into the information filtering network model in sequence and perform one iteration.

[0051] In step S3-2, after each iteration (i.e., the training set audio and video pass through the model), the model parameters of the last layer are used to calculate the loss error respectively, and then the loss error is back-propagated to update the model parameters.

[0052] Step S3-3, repeat steps S3-1 to S3-2 until the training completion condition is met, and obtain the trained information filtering network model.

[0053] The training completion conditions for model training are the same as those for conventional deep learning network models, that is, training is completed after the model parameters of each layer converge.

[0054] After the above iterative training and error calculation and back propagation during the iterative process, a trained information filtering network can be obtained. In this embodiment, the trained model is tested on the generated test set.

[0055] Step S4: input the posts to be classified into the trained information filtering network model to obtain classification results.

[0056] The experiment uses standard evaluation indicators for multi-label classification tasks, specifically weighted average F1 score, Hamming loss and Jaccard index to evaluate the performance of the network.

[0057] The F1 score is the harmonic mean of the precision and recall scores. A weighted average is used to calculate the individual F1 scores for each tag and then sum them, using weights relative to the number of posts for each tag. To estimate the error rate in classification, the Hamming loss function is used to calculate the proportion of incorrectly predicted tags among all predicted tags. Smaller values ​​indicate better performance. To evaluate the accuracy of the system, the Jaccard index is used to assess the similarity between predicted and true tags.

[0058] This experiment compares the information filtering network with other networks in multi-label classification using various evaluation metrics. TextCNN, HAN, BiLSTM, and MAGNET networks were selected for comparison. The hyperparameters of these networks were set according to the paper. To ensure fairness in the evaluation, all models were trained using the same training dataset and evaluated using the same test dataset.

[0059] This example uses the TREC-IS dataset and adopts the definition of actionable information from the authors of the TREC-IS challenge. The results of each model on the TREC-IS dataset and the COVID-19 post dataset are summarized in Table 1 below. The weighted average F1 score is used as the primary comparison metric. The weighted average F1 score considers the average performance of each network across all information types.

[0060] Table 1 shows the experimental results of different models on the TREC-IS dataset and the COVID-19 post dataset.

[0061] Table 1

[0062]

[0063]

[0064] In Table 1, the information filtering network outperforms the results of other networks in most indicators. In particular, the information filtering network has a weighted average F1 score that is 6 percentage points higher than the magnet network on the TREC-IS dataset and 4 percentage points higher than the magnet network on the COVID-19 post dataset.

[0065] In further analysis, the Jaccard index and Hamming loss were used to evaluate accuracy and error rate. The Jaccard index of the Information Filtering Network outperformed other networks on both datasets. The Information Filtering Network achieved a Jaccard index of 0.43 on both datasets, while the previous best MAGNET network achieved 0.38 on the TREC-IS dataset and 0.40 on the COVID-19 post dataset. On the other hand, the Information Filtering Network performed second only to the best HAN network using Hamming loss as the evaluation loss. On the TREC-IS dataset, the Information Filtering Network achieved the best performance, reaching 0.07. On the COVID-19 post dataset, the Information Filtering Network achieved a second-best performance of 0.08, compared to the best HAN network result of 0.04.

[0066] Experiments show that the information filtering network performs quite well in classifying disaster-related tweets into multiple information types. This is due to: (1) constructing a BERT encoder to capture the contextual information in tweets using contextual embeddings. (2) By injecting label information and using a graph attention network to integrate additional structural information between the post tags and labels, the semantics of the post representation is enriched. (3) Finally, a relational network is used to automatically learn the similarity between tweets and labels. By using a learnable distance function, an effective metric is learned using supervised learning to promote the mapping between tweets and multi-label outputs. In this embodiment, a meta-learning approach is used to learn the mapping between input features and multi-label outputs.

[0067] Since the ultimate goal is to evaluate the network's effectiveness in identifying posts with actionable information—that is, if an input tweet contains actionable information, the system should trigger an alert—the experiment also uses a new evaluation metric introduced by TREC-IS, called the cumulative alert value, to assess the system's ability to detect actionable information during a crisis. In this example, the cumulative alert value is used to evaluate the performance of the information filtering network model in identifying actionable information in tweets.

[0068] The cumulative alert value score ranges from -1 to +1, with positive values ​​indicating high information importance in the tweet and negative values ​​indicating low information importance. The experiment chose to define actionable information based on information type. The experimental results using the cumulative alert value are shown in Table 2. The first four rows show the evaluation results of other networks for multi-label classification. The last row shows the evaluation results of the information filtering network.

[0069] Table 2 shows the experimental results of different models using cumulative alarm values.

[0070] Table 2

[0071]

[0072] The experimental results in Table 2 show that the Information Filtering Network significantly outperforms all other networks. In particular, in the high-priority cumulative alert score, the Information Filtering Network achieves a 26 percentage point improvement over the MAGNET network model. Furthermore, the Information Filtering Network is the only network to achieve a positive cumulative alert score for high priorities.

[0073] This embodiment also provides a disaster information filtering system based on a graph attention network. The graph attention network is used to understand the correlation between words in a post and the corresponding information type, and filter to obtain actionable information, including:

[0074] The preprocessing unit performs preprocessing using the method in step S1 of this embodiment.

[0075] The information filtering unit performs information filtering using the method in steps S1 to S4 of this embodiment to obtain classification results.

[0076] Functions and Effects of the Embodiments

[0077] According to the disaster information filtering method based on graph attention network involved in this embodiment, the graph attention network is used to understand the correlation between the words of the post and the corresponding information type, and executable information is filtered, because it includes the following steps: Step 1, preprocessing the posts in the data set to obtain preprocessed data; Step 2, building an information filtering network model; Step 3, inputting the preprocessed data into the information filtering network model for training to obtain a trained information filtering network model; Step 4, inputting the posts to be classified into the trained information filtering network model to obtain classification results. Among them, the information filtering network model includes a BERT encoder, a graph attention network and a relational network.

[0078] Therefore, this example uses a model that combines the BERT encoder and the audio-graph attention network. This model integrates the contextual representation of tweets with the correlation between tweets and their tags. This allows for automatic classification of tweets into multi-label information types and filtering key information from massive social media data.

[0079] In addition, the disaster information filtering method and system based on graph attention network of the present invention are particularly suitable for use in social platforms.

[0080] The above embodiments are preferred examples of the present invention and are not intended to limit the scope of protection of the present invention.

Claims

1. A disaster information filtering method based on graph attention network, characterized in that: Using a graph attention network to understand the correlation between words in a post and the corresponding information type, and filtering to obtain actionable information, includes the following steps: Step 1: Preprocess the posts in the dataset to obtain preprocessed data. A marker is added to the beginning of the post and a marker is added after each sentence in the post as an indication of the sentence boundary. Each marker is divided into three types of embeddings: tagging, segmentation, and position embedding. Step 2: Building an information filtering network model, wherein the information filtering network includes a BERT encoder, a graph attention network, and a relational network. The BERT encoder sums the three embeddings into a single low-dimensional vector that captures the semantics of the post. The graph attention network learns structural information between nodes based on the relative importance of adjacent nodes, then extracts the learned label representation and concatenates it with the vector of the post as input to the relational network. The relational network uses a learnable distance metric and supervised learning to learn a similarity measure between the post vector and the label vector. Step 3: inputting the pre-processed data into the information filtering network model for training to obtain a trained information filtering network model; Step 4: Input the post to be classified into the trained information filtering network model to obtain a set of tags related to the post as a classification result.

2. The disaster information filtering method based on graph attention network according to claim 1 is characterized in that: in, In step 2, the information filtering network model is trained on the TREC-IS dataset and adopts the definition of actionable information from the TREC-IS challenge authors.

3. The disaster information filtering method based on graph attention network according to claim 1 is characterized in that: in, In step 2, the cumulative alert value is used to evaluate the performance of the information filtering network model to identify actionable information in tweets.

4. The disaster information filtering method based on graph attention network according to claim 1 is characterized in that: in, In step 2, meta-learning is used to learn the mapping between input features and multi-label outputs.

5. The disaster information filtering method based on graph attention network according to claim 1 is characterized in that: in, Step S3 includes the following steps: Step S3-1, inputting the pre-processed data into the information filtering network model in sequence and performing one iteration; Step S3-2, after the iteration, using the model parameters of the last layer to respectively calculate the loss error, and then backpropagating the loss error to update the model parameters; Step S3-3, repeat steps S3-1 to S3-2 until the training completion condition is met, and obtain the trained information filtering network model.

6. A disaster information filtering system based on graph attention network, characterized in that: Use graph attention networks to understand the correlation between a post’s words and the corresponding information type, filtering to obtain actionable information, including: A preprocessing unit preprocesses the posts in the dataset to obtain preprocessed data, wherein a marker is added to the beginning of the post and a marker is added after each sentence in the post as an indication of a sentence boundary. Each marker is divided into three types of embeddings, namely, tagging, segmentation, and position embedding; The information filtering unit builds an information filtering network model, inputs the pre-processed data into the information filtering network model for training, obtains a trained information filtering network model, inputs the posts to be classified into the trained information filtering network model, and obtains a set of tags related to the posts as classification results. Among them, the information filtering network includes a BERT encoder, a graph attention network and a relational network. The BERT encoder adds the three embeddings into a single low-dimensional vector, which captures the semantics of the post; the graph attention network learns the structural information between nodes based on the relative importance of adjacent nodes, and then extracts the learned label representation and connects it with the vector of the post as the input of the relational network; the relational network uses a learnable distance metric and supervised learning to learn the similarity measure between the vector of the post and the label vector.

Images