A GCN-based method for extracting trigger words for agricultural meteorological disaster events
Through the GCN-based method, combined with the BERT model and the bidirectional long and short-term memory network, a dependency syntax graph and word-long mask matrix are constructed, which solves the problems of low information acquisition efficiency of agricultural meteorological disaster events and mismatch of trigger word boundaries, significantly improving the event The accuracy and efficiency of the extraction provide efficient technical support for agricultural meteorological disaster warning.
Patent Information
- Application Number
- CN202411114001.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The acquisition and processing of agricultural meteorological disaster events is inefficient. The boundary mismatch of trigger words and the polysense of the word in Chinese text leads to the high complexity of event extraction, affecting the accuracy of disaster warning.
Using GCN-based method, through data preprocessing, sentence encoding, graph convolution network and bidirectional long and short-term memory network, a dependent syntax graph and word-long mask matrix are constructed, combined with the BERT model to capture deep semantic information, and event-triggered word extraction and Classification.
It improves the accuracy and efficiency of event extraction, reduces the complexity of graph construction, solves the problems of trigger word mismatch and multiple meanings of one word, and improves the accuracy of agricultural meteorological disaster warning.
Smart Images
Figure CN118862900B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of meteorological forecasting, and in particular to a method for extracting trigger words of agricultural meteorological disaster events based on GCN. Background Art
[0002] As a basic industry of the national economy, agricultural production activities are deeply affected by various factors, among which meteorological disasters are one of the most serious factors. Therefore, improving the disaster resistance of agricultural production and reducing disaster losses are of great significance to ensuring food security.
[0003] An event is a specific form of information representation that describes objective facts that occur at a specific time and place. As one of the subtasks of information extraction, event extraction has the core task of automatically identifying specific types of events and their arguments from unstructured texts to form a structured representation, which is the basis of decision-making systems and knowledge graphs.
[0004] At present, the information acquisition and processing of agricultural meteorological disaster events are facing new challenges. With the rapid development of Internet technology, a large amount of agricultural meteorological disaster information is presented in an unstructured form. The exponential growth of information volume has led to a decrease in the efficiency of extracting effective information. At the same time, a large amount of agricultural meteorological data has not been fully utilized, and the disaster event information contained in it has not been effectively mined. Therefore, applying event extraction technology to the field of agricultural meteorology can achieve rapid identification and analysis of disaster events, and provide timely warning and decision support for agricultural production.
[0005] Event extraction tasks usually include two subtasks: event detection and argument extraction. Event detection aims to identify trigger words and classify them into corresponding event types, while argument extraction focuses on identifying event arguments and the roles they play.
[0006] Due to the lack of natural separators in Chinese text, there is a problem that the trigger word boundary does not match the word boundary, which may cause the event detection method based on trigger words to easily make classification errors. In addition, event trigger words have polysemous meanings, and different word segmentations may produce different semantics, which increases the complexity of event extraction.
[0007] Therefore, it is necessary to design a GCN-based agricultural meteorological disaster event trigger word extraction method, aiming to explore an efficient and accurate event trigger word extraction method to provide timely early warning and decision support for agricultural production, which is of great significance to improving the accuracy of agricultural meteorological disaster early warning. Summary of the invention
[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a GCN-based agricultural meteorological disaster event trigger word extraction method, aiming to explore an efficient and accurate event trigger word extraction method to provide timely early warning and decision support for agricultural production, which is of great significance to improving the accuracy of agricultural meteorological disaster early warning.
[0009] In order to achieve the above object, the present invention provides a trigger word extraction method for agricultural meteorological disaster events based on GCN:
[0010] The following steps are involved:
[0011] S1, data preprocessing:
[0012] S1-1, collect text data related to agricultural meteorological disasters from various data sources, and clean the text data to remove irrelevant information;
[0013] S1-2, processing the cleaned agricultural meteorological disaster text data into sentences, so that each sentence is used as an independent processing unit;
[0014] S1-3, use spacy tool to perform word segmentation and tag each word with part of speech;
[0015] S1-4, use spacy tool to perform dependency syntactic analysis, generate a dependency syntactic tree for each sentence, and show the dependency relationship between words;
[0016] S2, sentence encoding module:
[0017] S2-1, for a given sentence X = (x1, x2, ..., x i ,…,x n ) where x i Represents the i-th word, obtains the word vector, position vector and segment vector of the word encoding of the given sentence;
[0018] The word vector represents the index of each word in the vocabulary;
[0019] Segment vectors are used to distinguish words between different sentences;
[0020] The position vector is used to indicate which parts of the input the model needs to focus on when processing it;
[0021] S2-2, input the word encoding into the pre-trained BERT model for processing, and encode the sentence into a semantically rich vector representation B;
[0022] S3, intermediate word vector generation module:
[0023] S3-1, using graph convolutional network GCN to extract event trigger words to capture complex dependencies and contextual information in text;
[0024] S3-2, using dependency syntactic analysis to build a sentence syntactic graph for building a graph convolutional network, specifically:
[0025] S3-2-1, use Spacy tool to segment the sentence to obtain the grammatical situation of the text; get the word set W = (w1, w2, ..., w i ,…,w m ), get the word length of each word, the word length indicates the number of characters contained in each word, and construct the word length mask matrix M through the word length, where M is an m×n matrix, where m is the word length of the sentence and n is the character length of the sentence. The elements M in the matrix M i,j Indicates whether the jth character belongs to the ith word. If so, the corresponding element value is M i,j =1, otherwise, M i,j =0;
[0026] S3-2-2, multiply the word length mask matrix M by the word embedding B output by BERT and take the average to convert the character-level embedding vector into a word-level embedding vector;
[0027] S3-2-3, use Spacy to map part-of-speech tags to embedding vectors POS, and combine part-of-speech embedding with word embedding. Connect them to form new word embeddings;
[0028] S4, graph convolutional network module:
[0029] S4-1, define the dependency syntactic graph G = (V, E), where V = (v1, v2, ..., v i ,…,v m ) contains m nodes, each node corresponds to a word, and E represents the set of edges between each node;
[0030] S4-2, if two nodes v belong to V i With v j If there is a dependency relationship between them, then there is an edge (v i ,v j ) is the positive syntactic edge of the node;
[0031] S4-3, add reverse syntactic edge (v j ,v i ) is used to ensure the reverse transmission of information, and on this basis, a self-loop edge (v i ,v i )Introduce the node's own information;
[0032] S4-4, the word embedding obtained by the intermediate word vector generation module is used as a node, and the adjacency matrix A is constructed based on the forward syntactic edges, reverse syntactic edges, and self-loop edges obtained by dependency syntactic analysis; if the node v i 、v jIf there is a dependency relationship, then A ij =1 is input into the GCN model;
[0033] S4-5, the calculation formula of the graph convolutional network is:
[0034]
[0035] Among them, H (l+1) represents the node feature matrix of the l+1 layer, A represents the adjacency matrix, is the degree matrix, σ is the activation function, H (l) represents the node feature matrix of layer l, W (l) Represents the weight matrix of the lth layer; the graph convolutional network updates the representation of the node by aggregating the features of adjacent nodes to further obtain semantic features;
[0036] S5, bidirectional long short-term memory network module:
[0037] After the GCN model obtains the word embedding, it uses the word length mask matrix to process the word embedding to generate the word embedding, and concatenates it with the output B of the BERT model to obtain the semantic information of the word, which is then input into the BiLSTM layer of the bidirectional long short-term memory network for sequence modeling;
[0038] The BiLSTM network captures the long-distance dependencies of sentences. The BiLSTM network converts word x i Encoded as:
[0039]
[0040] Among them, x t represents the input features at time step t, represents the hidden state of the forward LSTM at time step t,
[0041] represents the hidden state of the reverse LSTM at time step t.
[0042] After BiLSTM encoding, word x i The code is Coming soon Put them together to get new Encoding vector
[0043] S6, trigger word identification and classification:
[0044] S6-1, the output of the BiLSTM model is passed to the intermediate linear layer for linear transformation, trigger word recognition is converted into a classification task, and the output of the linear layer is converted into a probability value P of 0 to 1 through the Sigmoid activation function;
[0045] S6-2, use the pointer network method to decode trigger words. For each token, calculate its probability value at the start position and end position of each event type, and set the threshold of the trigger word start index and end index. When p is greater than the threshold, it is marked as the start or end position of the candidate trigger word, and the candidate trigger word with the largest p is determined as the final trigger word.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] The present invention utilizes the large model BERT to generate word vectors with rich semantic information, and converts the character embeddings output by BERT into word embeddings by constructing a word length mask matrix, thereby efficiently realizing the conversion of character embeddings to word embeddings, reducing the complexity of graph construction, and improving event extraction efficiency.
[0048] At the same time, the present invention utilizes the advantages of GCN in modeling complex relationships and semantic information to solve the problems of mismatch and polysemy of trigger words in Chinese event extraction, thereby improving the accuracy of event extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a data preprocessing flow chart of the present invention.
[0050] Figure 2 A flow chart for generating the intermediate word vector of the present invention.
[0051] Figure 3 Schematic diagram of an example of dependency syntactic arc of the present invention.
[0052] Figure 4 This is a flow chart of trigger word recognition and classification of the present invention.
[0053] Figure 5 The figure is a flow chart of the method for extracting trigger words for meteorological disaster events according to the present invention. DETAILED DESCRIPTION
[0054] See also Figures 1 to 5 , the present invention is further described in conjunction with the accompanying drawings:
[0055] The present invention provides a method for extracting trigger words of agricultural meteorological disaster events based on GCN. Figure 5 As shown, it specifically includes the following steps:
[0056] S1, such as Figure 1 As shown, data preprocessing:
[0057] S1-1, collect text data related to agricultural meteorological disasters from various data sources, and clean the text data to remove irrelevant information;
[0058] S1-2, processing the cleaned agricultural meteorological disaster text data into sentences, so that each sentence is used as an independent processing unit;
[0059] S1-3, use spacy tool to perform word segmentation and tag each word with part of speech;
[0060] S1-4, use spacy tool to perform dependency syntactic analysis, generate a dependency syntactic tree for each sentence, and show the dependency relationship between words;
[0061] S2, sentence encoding module:
[0062] S2-1, for a given sentence X = (x1, x2, ..., x i ,…,x n ), where x i Represents the i-th word, obtains the word vector, position vector and segment vector of the word encoding of the given sentence;
[0063] The word vector represents the index of each word in the vocabulary;
[0064] Segment vectors are used to distinguish words between different sentences;
[0065] The position vector is used to indicate which parts of the input the model needs to focus on when processing it;
[0066] S2-2, input the word encoding into the pre-trained BERT model for processing, and encode the sentence into a semantically rich vector representation B;
[0067] S3, intermediate word vector generation module:
[0068] S3-1 uses graph convolutional network (GCN) to extract event trigger words to capture complex dependencies and contextual information in text. Through multi-layer convolution operations, GCN can fuse the local information of a node with the information of its neighboring nodes to obtain global contextual information.
[0069] S3-2, using dependency syntactic analysis to build a sentence syntactic graph for building a graph convolutional network, specifically:
[0070] S3-2-1, use Spacy tool to segment the sentence to obtain the grammatical situation of the text; get the word set W = (w1, w2, ..., w i ,…,w m ), get the word length of each word, the word length indicates the number of characters contained in each word, and construct the word length mask matrix M through the word length, where M is an m×n matrix, where m is the word length of the sentence and n is the character length of the sentence. The elements M in the matrix M i,jIndicates whether the jth character belongs to the ith word. If so, the corresponding element value is M i,j =1, otherwise, M i,j =0;
[0071] S3-2-2, multiply the word length mask matrix M by the word embedding B output by BERT and take the average to convert the character-level embedding vector into a word-level embedding vector;
[0072] S3-2-3, such as Figure 2 As shown in the figure, the Spacy tool is used to map the part-of-speech tags to embedding vectors POS, and the part-of-speech embeddings are concatenated with the word embeddings to form new word embeddings. This method not only efficiently realizes the conversion from character-level embedding to word-level embedding, but also integrates the semantic and grammatical information of the original text to generate a richer text representation.
[0073] S4, graph convolutional network module:
[0074] S4-1, define the dependency syntactic graph G = (V, E), where V = (v1, v2, ..., v i ,…,v m ) contains m nodes, each node corresponds to a word, and E represents the set of edges between each node;
[0075] S4-2, if two nodes v belong to V i With v j If there is a dependency relationship between them, then there is an edge (v i ,v j ) is the positive syntactic edge of the node;
[0076] S4-3, add reverse syntactic edge (v j ,v i ) is used to ensure the reverse transmission of information, and on this basis, a self-loop edge (v i ,v i )Introduce the node's own information;
[0077] For example, in the sentence "Drought occurred in Hebei Province", there is a syntactic arc nsubj from "appeared" to "Hebei Province", and there is a syntactic arc dobj from "appeared" to "drought", such as Figure 3 As shown, (appearance, Hebei Province) has a positive syntactic edge, and (appearance, drought) has a positive syntactic edge.
[0078] S4-4, the word embedding obtained by the intermediate word vector generation module is used as a node, and the adjacency matrix A is constructed based on the forward syntactic edges, reverse syntactic edges, and self-loop edges obtained by dependency syntactic analysis; if the node v i 、v j If there is a dependency relationship, then A ij=1 is input into the GCN model;
[0079] S4-5, the calculation formula of the graph convolutional network is:
[0080]
[0081] Among them, H (l+1) represents the node feature matrix of the l+1 layer, A represents the adjacency matrix, is the degree matrix, σ is the activation function, H (l) represents the node feature matrix of layer l, W (l) represents the weight matrix of the lth layer;
[0082] The graph convolutional network updates the representation of nodes by aggregating the features of adjacent nodes to further obtain semantic features;
[0083] S5, bidirectional long short-term memory network module:
[0084] After the GCN model obtains the word embedding, it uses the word length mask matrix to process the word embedding to generate the word embedding, and concatenates it with the output B of the BERT model to obtain the semantic information of the word, which is then input into the BiLSTM layer of the bidirectional long short-term memory network for sequence modeling;
[0085] The BiLSTM network captures the long-distance dependencies of sentences. The BiLSTM network converts word x i Encoded as:
[0086]
[0087] Among them, x t represents the input features at time step t, represents the hidden state of the forward LSTM at time step t, represents the hidden state of the reverse LSTM at time step t.
[0088] After BiLSTM encoding, word x i The code is Coming soon Concatenate them together to get a new encoding vector
[0089] S6, such as Figure 4 As shown, trigger word identification and classification:
[0090] S6-1, the output of the BiLSTM model is passed to the intermediate linear layer for linear transformation, trigger word recognition is converted into a classification task, and the output of the linear layer is converted into a probability value P of 0 to 1 through the Sigmoid activation function;
[0091] S6-2, use the pointer network method to decode trigger words. For each token, calculate its probability value at the start position and end position of each event type, and set the threshold of the trigger word start index and end index. When p is greater than the threshold, it is marked as the start or end position of the candidate trigger word, and the candidate trigger word with the largest p is determined as the final trigger word.
[0092] The above are only preferred embodiments of the present invention, which are only used to help understand the method and core ideas of the present application. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
[0093] The above embodiments can be implemented in whole or in part by software or hardware. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
[0094] The present invention solves the shortcomings of the prior art that agricultural meteorological data are not fully utilized, the disaster event information contained therein is not effectively mined, and the phenomenon of polysemy in Chinese vocabulary leads to different semantics in different word segmentations, which increases the complexity of event extraction. By utilizing BERT to capture the deep semantic information of the text and a new graph construction method, the character embedding output by BERT is converted into a word embedding by constructing a word length mask matrix, and a new word embedding is generated by combining the part-of-speech embedding. The word embedding is passed into the GCN network as a node to mine the long-distance dependency relationship within the sentence, and the text sequence is further processed in combination with BiLSTM, and then the trigger word is decoded through the pointer network, which significantly improves the accuracy and efficiency of trigger word extraction, and provides efficient technical support for agricultural meteorological disaster management.
Claims
1. A GCN-based method for extracting trigger words for agricultural meteorological disaster events, characterized in that: The following steps are involved: S1, data preprocessing: S1-1, collect text data related to agricultural meteorological disasters from various data sources, and clean the text data to remove irrelevant information; S1-2, processing the cleaned agricultural meteorological disaster text data into sentences, so that each sentence is used as an independent processing unit; S1-3, use spacy tool to perform word segmentation and tag each word with part of speech; S1-4, use spacy tool to perform dependency syntactic analysis, generate a dependency syntactic tree for each sentence, and show the dependency relationship between words; S2, sentence encoding module: S2-1, for a given sentence ,in Represents the i-th word, obtains the word vector, position vector and segment vector of the word encoding of the given sentence; The word vector represents the index of each word in the vocabulary; The segment vector is used to distinguish words between different sentences; The position vector is used to indicate which parts the model needs to focus on when processing the input; S2-2, inputting the word encoding into the pre-trained BERT model for processing, encoding the sentence into a semantically rich vector representation B; S3, intermediate word vector generation module: S3-1, using graph convolutional network GCN to extract event trigger words to capture complex dependencies and contextual information in text; S3-2 uses dependency syntactic analysis to construct a syntactic graph of the sentence, which is used to build a graph convolutional network. Specifically: S3-2-1, use Spacy tool to segment the sentence to obtain the grammatical situation of the text; get the word set , get the word length of each word, the word length indicates the number of characters contained in each word, and construct the word length mask matrix M through the word length, where M is an m×n matrix, where m is the word length of the sentence and n is the character length of the sentence; the element M in the matrix M i,j Indicates whether the jth character belongs to the ith word. If so, the corresponding element value is M i,j =1, otherwise, M i,j =0; S3-2-2, multiplying the word length mask matrix M by the word embedding B output by the BERT and taking the average, converting the character-level embedding vector into a word-level embedding vector; S3-2-3, use the Spacy tool to map the part-of-speech tag to an embedding vector POS, and concatenate the part-of-speech embedding with the word embedding to form a new word embedding; S4, graph convolutional network module: S4-1, define dependency syntax graph ,in It contains m nodes, each node corresponds to a word, and E represents the set of edges between each node; S4-2, if two nodes belonging to V and If there is a dependency relationship between them, then there is an edge is the positive syntactic edge of the node; S4-3, add reverse syntactic edge , which is used to ensure the reverse transmission of information, and on this basis, add a self-loop edge to each node Introduce the node's own information; S4-4, using the word embedding obtained by the intermediate word vector generation module as a node, constructing the adjacency matrix A based on the forward syntactic edges, reverse syntactic edges, and self-loop edges obtained by dependency syntactic analysis; if the node , If there is a dependency relationship, Input into the GCN model; S4-5, the calculation formula of the graph convolutional network is: ； in, represents the node feature matrix of the l+1 layer, A represents the adjacency matrix, is the degree matrix, , is the activation function, represents the node feature matrix of layer l, represents the weight matrix of the lth layer; the graph convolutional network updates the representation of the node by aggregating the features of adjacent nodes to further obtain semantic features; S5, bidirectional long short-term memory network module: After the GCN model obtains the word embedding, the word embedding is processed using the word length mask matrix to generate a character embedding, and the word embedding is concatenated with the output B of the BERT model to obtain the semantic information of the word, which is then input into the bidirectional long short-term memory network BiLSTM layer for sequence modeling; The BiLSTM network captures the long-distance dependencies of sentences. Encoded as: ； ； in, represents the input features at time step t, represents the hidden state of the forward LSTM at time step t, represents the hidden state of the reverse LSTM at time step t; After BiLSTM encoding, the word The code is , will soon Concatenate them together to get a new encoding vector ; S6, trigger word identification and classification: S6-1, the output of the BiLSTM model is passed to the intermediate linear layer for linear transformation, trigger word recognition is converted into a classification task, and the output of the linear layer is converted into a probability value P of 0 to 1 through the Sigmoid activation function; S6-2, use the pointer network method to decode trigger words. For each token, calculate its probability value p at the start position and end position of each event type, and set the threshold of the trigger word start index and end index. When p is greater than the threshold, it is marked as the start or end position of the candidate trigger word, and the candidate trigger word with the largest p is determined as the final trigger word.