Chinese event detection method based on part-of-speech attention mechanism

Abstract

The application provides a Chinese event detection method based on a part-of-speech attention mechanism, which is based on a public data set. First, the sentence is divided into words using a word segmentation tool on the data set, and then the part-of-speech sequence of the sentence is obtained using a part-of-speech tagging tool. The part-of-speech sequence of the sentence is input into a CBOW model to obtain a pre-trained part-of-speech vector, so as to learn the fixed collocation information between words, such as "suffered an injury" as a "verb + adverb + noun" structure. Then, the part-of-speech vector, the word vector and the character vector are used to extract the word-level information and the character-level information of the sentence, respectively. When extracting features, a convolutional neural network is used to extract features on the word matrix, the character matrix and the part-of-speech matrix of the sentence. Then, the part-of-speech features are used to calculate the attention score, which is used to assist the model to focus on the verb when calculating; the part-of-speech attention is provided, and after the part-of-speech features are added, the accuracy and efficiency of the model in the trigger word extraction and event type classification tasks are higher.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, specifically relating to a Chinese event detection method based on part-of-speech attention mechanism. Background Technology

[0002] Event extraction is one of the tasks of information extraction, primarily involving the extraction of structured textual information from unstructured text. The role of event extraction in daily life is also significant. In the financial field, extracting financial events from the stock market helps investors and corporate decision-makers understand stock market fluctuations and make more informed decisions. For government departments, extracting information from trending news events helps them conduct public opinion monitoring and understand the development of these events. In natural language processing (NLP) research, the structured data obtained through event extraction can be used to construct knowledge graphs and for other NLP tasks such as question answering and information retrieval. According to the definition of the ACE (Automatic Content Extraction) conference, existing research divides event extraction into four tasks: trigger word detection, event type or trigger word type identification, event argument detection, and argument role classification.

[0003] Existing event detection methods can be summarized into two types: one introduces external knowledge bases to improve model performance, and the other provides new models or improves existing models to fully mine various information for event detection tasks. The model performance for event detection tasks is not ideal in either case.

[0004] At present, most deep learning-based models focus on utilizing semantic role information, dependency information, and pre-trained models, neglecting the importance of part-of-speech tags. This results in unsatisfactory model performance in event detection tasks, with low efficiency and accuracy in extracting event trigger words and classifying their event types.

[0005] Therefore, there is a current need for an event detection method that can quickly extract trigger words and classify events. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a Chinese event detection method based on part-of-speech attention mechanism.

[0007] To achieve the technical effect of solving the above-mentioned technical problems, the present invention is implemented through the following technical solution: a Chinese event detection method based on part-of-speech attention mechanism, characterized by comprising the following steps:

[0008] Step 1: Use a word segmentation tool to segment the sentence, and then use a part-of-speech tagging tool to obtain the part-of-speech tag for each word. After obtaining the part-of-speech tags, use the word2vec method to obtain the pre-trained part-of-speech tag vector.

[0009] Step 2: Obtain the part-of-speech feature matrix of the sentence, and use a convolutional neural network to extract part-of-speech features;

[0010] Step 3: Use a word segmentation tool to segment the sentence and obtain the word vector matrix and character vector matrix of the sentence; then use a convolutional neural network on the word matrix and sentence matrix of the sentence to extract the vocabulary-level features and character-level features of the sentence respectively;

[0011] Step 4: Multiply the vocabulary-level features and part-of-speech attention, and then concatenate the vocabulary-level features with the features of the context words to obtain new vocabulary-level features;

[0012] Step 5: The word-level features and character-level features are concatenated to obtain the input features for the block extraction network, which then outputs the block classification label for the word.

[0013] Step 6: Calculate the loss using the output labels and real labels of the block extraction network, and optimize the word convolutional network, character convolutional network, part-of-speech attention module, and block extraction network using Adadelta backpropagation;

[0014] Furthermore, in Step 2, convolution kernels are used to perform convolution calculations on words, extract part-of-speech features, concatenate the convolution features into a part-of-speech feature matrix, and use fully connected layers and softmax layers to calculate part-of-speech attention.

[0015] Furthermore, in Step 3, the sentence is divided into words and characters for processing, and word-level features and character-level features are extracted to obtain vocabulary-level features and character-level features;

[0016] Furthermore, in Step 4, the word vectors of the context words in the current word window of 2 are concatenated as features, and a convolutional network is used to extract the context part-of-speech features; and the part-of-speech attention score is multiplied by the part-of-speech features and concatenated with the context part-of-speech features as word-level features;

[0017] Furthermore, Step 5 specifically includes the following steps:

[0018] Step 5.1: After aggregating the word-level features and character-level features into a feature vector using a multilayer perceptron, the vectors are concatenated to trigger word extraction and event type classification;

[0019] Step 5.2: After concatenating the word-level features and character-level features, input them into the trigger word block extraction network to classify the trigger word blocks of the current character;

[0020] Step 5.3: After the trigger word extraction module extracts the trigger word, the hybrid feature corresponding to the trigger word is input into the event type classification network;

[0021] The beneficial effects of this invention are:

[0022] This invention first uses an NLP word segmentation tool to obtain part-of-speech (POS) sequences, then uses the word2vec method to obtain POS embedding vectors. A convolutional network is then used to extract POS features of the current word, character features of the sentence, word features of the sentence, and context word features. Next, POS attention is calculated using the POS features, and then multiplied by the word-level features using the POS attention. The word-level features and context word features are concatenated to obtain new word-level features, which are then input into a block extraction network to extract trigger words and into an event classification network to identify event types. The model achieves higher accuracy and efficiency in both trigger word extraction and event type classification tasks. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is an overall flowchart of a Chinese event detection method based on part-of-speech attention mechanism. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1

[0027] A Chinese event detection method based on part-of-speech attention mechanism includes the following steps:

[0028] Step 1: Segment the sentence using the pyltp word segmentation tool, then use a part-of-speech tagging tool to obtain the part-of-speech tag for each word. After obtaining the part-of-speech tags, use the word2vec method to obtain a pre-trained part-of-speech tag vector. Use this part-of-speech tag vector as the pre-trained part-of-speech embedding R.

[0029] Step 1.1: Input the part of speech of the context. , , , The model outputs the part-of-speech tag of the current word as a label to calculate the loss and optimize the model.

[0030] (1)

[0031] (2)

[0032] in, One-hot encoding of part-of-speech tags V represents the size of the part-of-speech dictionary. It is a part-of-speech vector matrix, where d is the vector dimension. Transform intermediate results into final output. Each element in the set represents the probability of outputting that part of speech. The objective function is to maximize the following conditional probability:

[0033] (3)

[0034] Step 2: Obtain the part-of-speech feature matrix of the sentence, and use a convolutional neural network to extract part-of-speech features. After passing through a fully connected layer, the part-of-speech features are mapped to a vector with the same length as the sentence. This vector is then processed by softmax to obtain the part-of-speech attention score P.

[0035] Step 2.1: Use k convolutional kernels to perform convolution calculations on h words, extracting part-of-speech features. Then, apply max pooling to the k channels, retaining the features extracted by one of the convolutional kernels. This represents the concatenation of h word vectors. This represents the weight, and b is the bias term. This represents the activation function.

[0036] (4)

[0037] (5)

[0038] Step 2.2: Concatenate the convolutional features of the n words into a part-of-speech (POS) feature matrix. POS attention is calculated using a fully connected layer and a softmax layer. Here, c is the POS feature matrix. This is a part-of-speech vector.

[0039] (6)

[0040] (7)

[0041] (8)

[0042] Step 3: Segment the sentence using the pyltp word segmentation tool, and obtain the word vector matrix W and character vector matrix C of the sentence using glove pre-training. Then, use convolutional neural networks on the word matrix and sentence matrix respectively to extract lexical-level features of the sentence. Character-level features .

[0043] Step 3.1: Using an algorithm similar to Step 2, the sentence is divided into words and characters for processing, and word-level features and word-level features are extracted to obtain lexical-level features. Character-level features .

[0044] Step 4: Analyze vocabulary-level features The part-of-speech attention P-dot product is used to make the model focus on verb information during computation, and then the lexical-level features and context word features are combined. New vocabulary-level features are obtained by splicing. .

[0045] Step 4.1: Concatenate the word vectors of the context words in the current word window of 2 as features, and also use a convolutional network to extract context part-of-speech features.

[0046] Step 4.2: Multiply the part-of-speech attention score by the part-of-speech feature and concatenate it with the context part-of-speech feature to obtain the word-level feature.

[0047] (9)

[0048] Step 5, and The concatenated features are used as input features for the block extraction network, which then outputs the block classification label for the word.

[0049] Step 5.1, and After being aggregated into feature vectors using a multilayer perceptron, these vectors are concatenated for trigger word extraction and event type classification.

[0050] (10)

[0051] Step 5.2: To avoid the trigger word mismatch problem caused by NLP word segmentation tools, this paper concatenates word-level features and character-level features and inputs them into the trigger word block extraction network to classify the trigger word blocks of the current character.

[0052] (11)

[0053] in dn adds a NIL tag to all possible combinations of trigger words. This is the weight matrix. This is a bias term.

[0054] Step 5.3: After the trigger word extraction module extracts the trigger word, it will extract the mixed features corresponding to the trigger word. The input is fed into an event type classification network, which uses a fully connected layer as a classifier.

[0055] (12)

[0056] in, This is the weight matrix. For bias terms, And dt=34 (33 event types plus a NIL label). After the fully connected layer outputs, softmax is used to obtain the score of the current trigger word into each category, and the category with the highest score is the event type of the trigger word.

[0057] Step 6: Calculate the loss using the output labels and real labels of the block extraction network, and optimize the word convolutional network, character convolutional network, part-of-speech attention module, and block extraction network using Adadelta backpropagation.

[0058] Example 2

[0059] like Figure 1 As shown, a Chinese event detection method based on a block extraction neural network incorporating a part-of-speech attention mechanism is described. The specific steps of the method are as follows:

[0060] Step 1: Use the pyltp word segmentation tool to divide the sentence into many words on the corpus, and then obtain the part-of-speech tags corresponding to these words. Treat these part-of-speech tags as words and use the CBOW model to pre-train part-of-speech vectors. The purpose of this operation is to make the part-of-speech vectors contain certain part-of-speech collocation information, such as "verb + adverb + noun".

[0061] Step 2: Input a new sentence, use a word segmentation tool to split the sentence into multiple words, then obtain the sentence's word matrix based on pre-trained GloVe word vectors, and use a convolutional neural network to extract features from the word matrix. Multiple convolutional kernels are used to extract sentence features, and then max pooling is used to retain the features extracted by one of the convolutions.

[0062] Step 3: After sentence segmentation, obtain part-of-speech (POS) tags. Then, based on the pre-trained POS tag vectors obtained in Step 1, obtain the POS matrix of the sentence. Use a convolutional network with the same structure as in Step 2 to extract the POS feature vectors. Use a mapping matrix to process the POS feature vectors into vectors of the same length as the sentence, and use softmax to obtain the POS attention score. Multiply this POS attention score by the POS feature vectors to filter the vocabulary-level features.

[0063] Step 4: Use Glove vectors to obtain a character matrix for all characters in the entire sentence, and use a convolutional network with the same structure as in Step 2 to extract character-level features.

[0064] Step 5: After concatenating the word-level features and sentence-level features, input them into the trigger word chunk extractor and event type classifier.

[0065] Step 6: Both the trigger word extractor and the event type classifier are fully connected networks, but they differ in the final mapped categories. The trigger word extractor classifies the events into 7 tags, such as... Figure 1 As shown, the event type classifier categorizes events into 34 event type labels.

[0066] In summary, this invention first uses an NLP word segmentation tool to obtain a part-of-speech sequence, then uses the word2vec method to obtain part-of-speech embedding vectors. A convolutional network is then used to extract the part-of-speech features of the current word, the character features of the sentence, the word features of the sentence, and the context word features. Next, part-of-speech attention is calculated using the part-of-speech features, and the part-of-speech attention is multiplied by the word-level features. The word-level features and context word features are then concatenated to obtain new word-level features. The concatenated word features and character features are then input into a block extraction network to extract trigger words, and into an event classification network to identify event types. This approach considers the importance of part-of-speech and provides part-of-speech attention. After incorporating part-of-speech features, the model achieves higher accuracy and efficiency in trigger word extraction and event type classification tasks.

[0067] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0068] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A Chinese event detection method based on part-of-speech attention mechanism, characterized in that, Includes the following steps: Step 1: Use a word segmentation tool to segment the sentence, and then use a part-of-speech tagging tool to obtain the part-of-speech tag for each word. After obtaining the part-of-speech tags, use the word2vec method to obtain the pre-trained part-of-speech tag vector. Step 2: Obtain the part-of-speech feature matrix of the sentence, and use a convolutional neural network to extract part-of-speech features; Step 3: Use a word segmentation tool to segment the sentence and obtain the word vector matrix and character vector matrix of the sentence; then use a convolutional neural network on the word matrix and sentence matrix of the sentence to extract the vocabulary-level features and character-level features of the sentence respectively; Step 4: Multiply the vocabulary-level features and part-of-speech attention, and then concatenate the vocabulary-level features with the features of the context words to obtain new vocabulary-level features; Step 5: The word-level features and character-level features are concatenated to obtain the input features for the block extraction network, which then outputs the block classification label for the word. Step 6: Calculate the loss using the output labels and real labels of the block extraction network, and optimize the word convolutional network, character convolutional network, part-of-speech attention module, and block extraction network using Adadelta backpropagation; In Step 4, the word vectors of the context words in the current word window of 2 are concatenated as features, and a convolutional network is used to extract the context part-of-speech features. The part-of-speech attention score is multiplied by the part-of-speech feature and then concatenated with the context part-of-speech feature to form a word-level feature; Step 5 specifically includes the following steps: Step 5.1: After aggregating the word-level features and character-level features into a feature vector using a multilayer perceptron, the vectors are concatenated to trigger word extraction and event type classification; Step 5.2: After concatenating the word-level features and character-level features, input them into the trigger word block extraction network to classify the trigger word blocks of the current character; Step 5.3: After the trigger word extraction module extracts the trigger word, the hybrid feature corresponding to the trigger word is input into the event type classification network.

2. The Chinese event detection method based on part-of-speech attention mechanism according to claim 1, characterized in that, In Step 2, convolution kernels are used to perform convolution calculations on words, extract part-of-speech features, concatenate the convolution features into a part-of-speech feature matrix, and use fully connected layers and softmax layers to calculate part-of-speech attention.

3. The Chinese event detection method based on part-of-speech attention mechanism according to claim 1, characterized in that, In Step 3, the sentence is divided into words and characters for processing, and word-level features and character-level features are extracted to obtain vocabulary-level features and character-level features.

Images