Chinese event extraction method and system based on word features and pointer network

Abstract

The invention provides a Chinese event extraction method and system based on word characteristics and a pointer network, and the method comprises the following steps: inputting an original Chinese sequence into a BERT model, carrying out segmentation and coding conversion to form a high-dimensional vector so as to realize word segmentation processing, and carrying out word segmentation processing on a text by adopting a jieba word segmentation tool; respectively inputting the character-level vector and the word-level vector into a BiLSTM network to carry out word feature extraction, and carrying out word feature fusion through an attention mechanism; and respectively identifying the initial positions of the trigger word and the argument by utilizing a double-sequence pointer network to realize extraction from the text. According to the Chinese event extraction method, the problems of insufficient semantic representation, role overlapping, argument nesting and the like in Chinese event extraction are solved, and the accuracy and generalization ability of an event extraction model are improved.

Description

Technical Field

[0001] The present invention relates to the field of event extraction, and in particular to a Chinese event extraction method and system based on word features and a pointer network. Background Art

[0002] Event extraction is one of the most challenging tasks in the field of information extraction. It refers to extracting events of interest to users from unstructured information and presenting them to users in a structured manner. Event extraction has a wide range of applications in event graph construction, public opinion analysis, information retrieval, information question and answer, etc., which has led to major breakthroughs in the development of natural language processing technology and is one of the strong technical supports for natural language processing in the field of engineering applications.

[0003] Different from English event element extraction, Chinese text has the characteristics of semantic ambiguity, complex word boundaries, and high-dimensional sparsity. Many event extraction algorithms in the English field cannot be directly applied to Chinese event extraction tasks. Existing Chinese event extraction methods have problems such as insufficient semantic representation, insufficient modeling of trigger words and argument dependencies when there is overlap in argument roles, which leads to poor argument extraction performance. These problems are important reasons for the difficulty of event extraction. How to fully and effectively utilize text features to improve the generalization and robustness of event extraction models is still a problem that needs to be solved.

[0004] In view of this, the present invention is proposed. Summary of the invention

[0005] In view of this, the present invention proposes a Chinese event extraction method and system based on word features and pointer network, constructs word vectors through the BERT model and designs a word-level feature extraction network at the same time, and uses BiLSTM to extract word-level and word-level features respectively, fuses the character vector with the weighted word vector, thereby obtaining a word feature fusion vector with more comprehensive and accurate semantic representation, inputs the fusion vector into the pointer network, and uses a double sequence pointer to identify the starting position of the trigger word and the argument respectively, making the decoding more flexible, effectively solving the problems of role overlap and argument nesting, and completing the final event extraction task. The purpose is to solve the problems of insufficient semantic representation, role overlap and argument nesting in Chinese event extraction, and improve the accuracy and generalization ability of the event extraction model.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] The present invention provides a Chinese event extraction method based on word features and pointer network, comprising the following steps:

[0008] The original Chinese sequence is input into the BERT model, segmented and encoded into a high-dimensional vector to achieve word segmentation, and the jieba word segmentation tool is used to segment the text;

[0009] The character-level vector and word-level vector are respectively input into the BiLSTM network to extract word features, and the word features are fused through the attention mechanism;

[0010] The dual sequence pointer network is used to identify the starting positions of trigger words and arguments respectively to achieve extraction from text.

[0011] In addition to providing a Chinese event extraction method based on word features and a pointer network, the present invention also provides a Chinese event extraction system based on word features and a pointer network, including:

[0012] Word segmentation module: It is used to input the original Chinese sequence into the BERT model, segment it and encode it into a high-dimensional vector to achieve word segmentation, and use the Jieba word segmentation tool to segment the text;

[0013] Word feature extraction and feature fusion module: input character-level vectors and word-level vectors into the BiLSTM network to extract word features, and perform word feature fusion through the attention mechanism;

[0014] Trigger word and argument extraction module: used to use the dual sequence pointer network to identify the starting position of the trigger word and argument respectively, and realize extraction from the text.

[0015] In summary, the solution of the present invention has the following beneficial effects:

[0016] (1) Although the BERT model segments Chinese characters by word, its deep learning feature enables it to capture the semantic relationship between characters and generate character vectors rich in semantic information. Therefore, the BERT model is used to segment and encode the input text, converting each character into a high-dimensional vector.

[0017] (2) In order to compensate for the vocabulary information that may be lost when BERT is segmented by words, the text is segmented using the Jieba word segmentation tool, and the word embedding vector is obtained through pre-training, which is then randomly initialized and optimized through model training.

[0018] (3) In order to effectively capture the contextual information of the sequence, this paper inputs the character-level vector and word-level vector processed by the embedding layer into BiLSTM respectively, uses BiLSTM to learn the deep context feature information in the input text, and organically integrates the character-level representation and word-level representation of the sequence through the attention mechanism to generate a multi-granularity mixed representation of the sequence, thereby making full use of the character-level semantic information of BERT and the word-level information of Jieba segmentation to obtain a more comprehensive and accurate semantic representation.

[0019] (4) In order to solve the problems of role overlap and argument nesting in Chinese event extraction, the decoding module of the model is constructed using a pointer network. The starting positions of trigger words and arguments are identified using dual sequence pointers respectively. The trigger word is regarded as a special argument and extracted from the event text together with other arguments. The argument roles are classified using a binary classification network model, thereby solving the problems of role overlap and argument nesting in event extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiments. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. The same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0021] Figure 1 It is the overall framework diagram of the Chinese event extraction method of the present invention;

[0022] Figure 2 It is a schematic diagram of character level feature extraction of the present invention;

[0023] Figure 3 It is a schematic diagram of word level feature extraction of the present invention;

[0024] Figure 4 A flowchart of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0026] The terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a", "said" and "the" used in the present disclosure and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" used herein may be interpreted as "at the time" or "when" or "in response to determining",

[0028] Example

[0029] like Figure 1 As shown, the Chinese event extraction method of the present invention provides a Chinese event extraction method based on word features and pointer network, comprising the following steps:

[0030] The original Chinese sequence is input into the BERT model, segmented and encoded into a high-dimensional vector to achieve word segmentation, and the jieba word segmentation tool is used to segment the text;

[0031] The character-level vector and word-level vector are respectively input into the BiLSTM network to extract word features, and the word features are fused through the attention mechanism;

[0032] The dual sequence pointer network is used to identify the starting positions of trigger words and arguments respectively to achieve extraction from text.

[0033] The above method is mainly implemented by three modules, namely, the character and word segmentation module based on character granularity and word granularity, the character feature extraction and feature fusion module based on BiLSTM, and the trigger word and argument extraction module based on the pointer network. The word vector is constructed through the BERT pre-training model and the word-level feature extraction network is designed at the same time. The character-level and word-level features are extracted by BiLSTM respectively, and the character vector is fused with the weighted word vector to obtain a more comprehensive and accurate word feature fusion vector with semantic representation. The fusion vector is input into the pointer network, and the starting position of the trigger word and the argument is identified respectively by using the double sequence pointer, so that the decoding is more flexible, the role overlap and argument nesting problems are effectively solved, and the final event extraction task is completed.

[0034] The specific implementation of the character and word segmentation module based on character granularity and word granularity follows the following steps:

[0035] When a text sequence is input into the BERT model, the hidden vector output by the last layer of Transformer Encoder of BERT is used as the feature representation of the character. The BERT model segments and encodes the input text, converting each character into a high-dimensional vector. n ], where w n is the vector representation of the nth character, and these vectors capture the contextual semantic information of the character. In order to make up for the vocabulary information that may be lost when BERT is segmented by word, this paper also uses the Jieba word segmentation tool to segment the text and count the word frequencies to build a vocabulary. Each word in the vocabulary is assigned a unique ID, including special symbols such as <pad>and <unk>, and mapped to the corresponding word embedding vectors. These word embedding vectors are obtained through pre-training, and then optimized through model training after random initialization.

[0036] The BiLSTM-based word feature extraction and feature fusion module is implemented according to the following modules:

[0037] In order to effectively capture the contextual information of the sequence, the character-level vector and word-level vector processed by the embedding layer are input into BiLSTM respectively, as follows: Figure 2 and Figure 3 As shown in the figure. By learning the deep feature information of the context in the input text and integrating the feature information output by the bidirectional long short-term memory network, the classification effect of the model is improved. i Represents the input text embedding, BiLSTM uses the forward LSTM network to calculate the state of the left part of the i-th character At the same time, a reverse LSTM network is used to calculate the state of the right part of the i-th character Then h i that is and The latent vector sequence after the two latent vectors are concatenated. H represents the result of merging the latent vector sequence from 1 to i, which is the output result of BiLSTM. The calculation process of the BiLSTM feature extraction layer can be formalized as follows:

[0038]

[0039]

[0040]

[0041] H=[h1,h2,…,h n ]

[0042] In order to make full use of the word-level semantic information of BERT and the word-level information of Jieba, the character vector generated by BERT is fused with the word embedding vector corresponding to the word segmentation result. For each character in the input text, find the word to which it belongs, and weight the word embedding vector of the word through the attention mechanism to reflect the importance of different words in the sentence. Assume that the vector after word embedding encoding is V = [v 1 ,v 2 ,…,v n ], the attention weight is calculated by the following formula:

[0043] [a 1 ,a 2 ,…,a n ]=softmax(linear(V))

[0044] To calculate the importance of each word, multiply and add the vector of each word and the corresponding weight, which can be expressed as follows:

[0045]

[0046] Get the weighted word vector V attn The weighted word embedding vector is concatenated with the character vector to obtain the fused word vector representation, denoted as H = [h1,h2,…,h n ], where h n is the fusion vector of the nth character, which contains both the contextual information of the character and the semantic information of the vocabulary. In this way, the embedding layer not only captures the character-level semantics of the text, but also enhances the understanding of complex semantics by introducing vocabulary information, providing a more comprehensive input for subsequent feature extraction and event extraction.

[0047] The trigger word and argument extraction module based on the pointer network is implemented according to the following steps:

[0048] There are a lot of role overlaps and argument nesting in Chinese event extraction tasks, which brings great challenges to event extraction methods. Confusion is easy to occur in the annotation and prediction stages, affecting the final recognition effect. Pointer network is a sequence-to-sequence neural network model that solves the problem of selecting specific elements from the input sequence and is widely used in various natural language processing tasks.

[0049] The pointer network consists of an encoder, a decoder and an attention mechanism. The encoder is usually a BiLSTM, which converts the input sequence into a hidden state vector, and each hidden state represents the characteristics of each element in the input sequence; the decoder is usually an LSTM, which generates a probability distribution pointing to a certain position in the input sequence at each step.

[0050] Here, dual sequence pointer recognition is used to locate the start and end positions of the trigger word and the argument in the input sequence respectively, and the trigger word is regarded as a special argument and extracted from the text together with the argument. Specifically, the annotations in the original dataset are defined as follows:

[0051] L=(E,Ti_s,Ti_e,R,Ar_s,Ar_e)

[0052] Among them, L represents an end-to-end label, E represents the event type, Ti_s and Ti_e represent the starting position and ending position of the trigger word respectively, R represents the argument role, Ar_s and Ar_e represent the starting position and ending position of the argument respectively. There are often multiple arguments in a single event text, and there are phenomena corresponding to multiple event types. Therefore, a single event text may have multiple end-to-end labels L, which are defined as follows:

[0053]

[0054] Among them, C is the complete label set corresponding to a single event text.

[0055] This module designs two pointers to locate the first and last positions of the trigger words and arguments in the event text sequence, and then classifies the argument roles using a binary classification network model. This module uses a pointer network through multiple rounds of decoding, cascaded pointers, and autoregressive output structures to flexibly handle the nested and hierarchical relationships between events, accurately capture nested events and roles, and effectively solve the problems of role overlap and argument nesting in event extraction.

[0056] In addition, in addition to providing a Chinese event extraction method, the present invention also provides a Chinese event extraction system, which specifically includes:

[0057] Word segmentation module: It is used to input the original Chinese sequence into the BERT model, segment it and encode it into a high-dimensional vector to achieve word segmentation, and use the Jieba word segmentation tool to segment the text;

[0058] Word feature extraction and feature fusion module: input character-level vectors and word-level vectors into the BiLSTM network to extract word features, and perform word feature fusion through the attention mechanism;

[0059] Trigger word and argument extraction module: used to use a dual sequence pointer network to identify the starting positions of trigger words and arguments respectively, and realize extraction from text.

[0060] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0061] The Chinese event extraction method of the present invention has the following beneficial effects:

[0062] Solve the problems of insufficient semantic representation, role overlap, and argument nesting in Chinese event extraction, and improve the accuracy and generalization ability of the event extraction model. Use the BERT pre-training model to build word vectors and design a word-level feature extraction network at the same time. Use BiLSTM to extract word-level and word-level features respectively, fuse the character vector with the weighted word vector, and obtain a word-word feature fusion vector with more comprehensive and accurate semantic representation. Input the fusion vector into the pointer network, and use the double sequence pointer to identify the starting position of the trigger word and argument respectively, making the decoding more flexible, effectively solving the problems of role overlap and argument nesting, and completing the final event extraction task.

[0063] Figure 4 This is a schematic diagram of the structure of a computer device disclosed in the present invention. Figure 4 As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403, and is used to store computer instructions executable by the processor 401, and the processor 401 is used to read computer instructions from the memory 402 to implement the steps of the method described in any of the above embodiments.

[0064] For the above-mentioned device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0065] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

[0066] Finally, it should be noted that although this specification contains many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of specific embodiments of specific inventions. Certain features described in multiple embodiments within this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claimed as such, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0067] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0068] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0069] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.< / unk> < / pad>

Claims

1. A Chinese event extraction method based on word features and pointer network, characterized in that: The steps include: The original Chinese sequence is input into the BERT model, segmented and encoded into a high-dimensional vector to achieve word segmentation, and the jieba word segmentation tool is used to segment the text; The character-level vector and word-level vector are respectively input into the BiLSTM network to extract word features, and the word features are fused through the attention mechanism; The dual sequence pointer network is used to identify the starting positions of trigger words and arguments respectively to achieve extraction from text.

2. The Chinese event extraction method according to claim 1, characterized in that: The method for character segmentation includes the following steps: using the hidden vector output by the last layer of the BERT model as the feature representation of the character, segmenting and encoding the input text, converting each character into a high-dimensional vector, and obtaining the input character vector W = [w1, w2, ..., w n ], where w n is the vector representation of the nth character.

3. The Chinese event extraction method according to claim 1, characterized in that: The method for word segmentation includes the following steps: Counting the frequency of words to build a vocabulary, each word in the vocabulary is assigned a unique ID, including special symbols such as <pad>and <unk> , and mapped to the corresponding word embedding vector.< / unk> < / pad> 4. The Chinese event extraction method according to claim 1, characterized in that: The method for organic fusion comprises the following steps: using a forward LSTM network to calculate the state of the left part of the i-th character At the same time, a reverse LSTM network is used to calculate the state of the right part of the i-th character Then h i that is and The latent vector sequence after the two latent vectors are concatenated, H represents the result of merging the latent vector sequence from 1 to i, which is the output result of BiLSTM. The specific formula is as follows: H=[h1,h2,…,h n ] where r i Represents the text embedding of the input; Assume that the vector after word embedding encoding is V = [v 1 ,v 2 ,…,v n ], the attention weight is calculated by the following formula: [a 1 ,a 2 ,…,a n ]=softmax(linear(V)) To calculate the importance of each word, multiply and add the vector of each word and the corresponding weight, which can be expressed as follows: Get the weighted word vector V attn ; The weighted word embedding vector is concatenated with the character vector to obtain the fused word vector representation, denoted as H = [h1,h2,…,h n ], where h n is the fused vector of the nth character.

5. The Chinese event extraction method according to claim 2, characterized in that: The dual sequence pointer network consists of an encoder, a decoder and an attention mechanism. The encoder is BiLSTM, which converts the input sequence into a hidden state vector; the decoder is LSTM, which generates a probability distribution pointing to a certain position in the input sequence at each step.

6. The Chinese event extraction method according to claim 1, characterized in that: The method of extracting from text by using a dual sequence pointer network to identify the starting positions of trigger words and arguments respectively includes the following steps: The annotations in the original dataset are as follows: L=(E,Ti_s,Ti_e,R,Ar_s,Ar_e) Among them, L represents an end-to-end label, E represents the event type, Ti_s and Ti_e represent the starting position and ending position of the trigger word respectively, R represents the argument role, Ar_s and Ar_e represent the starting position and ending position of the argument respectively, which are defined as follows: Among them, C is the total label set corresponding to a single event text.

7. The Chinese event extraction system of the Chinese event extraction method according to any one of claims 1 to 6, characterized in that: include: Word segmentation module: It is used to input the original Chinese sequence into the BERT model, segment it and encode it into a high-dimensional vector to achieve word segmentation, and use the Jieba word segmentation tool to segment the text; Word feature extraction and feature fusion module: input character-level vectors and word-level vectors into the BiLSTM network to extract word features, and perform word feature fusion through the attention mechanism; Trigger word and argument extraction module: used to use a dual sequence pointer network to identify the starting positions of trigger words and arguments respectively, and realize extraction from text.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the Chinese event extraction method as claimed in any one of claims 1 to 6 are executed.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the Chinese event extraction method as described in any one of claims 1-6 are implemented.

Images