A Chinese event extraction method and system based on word features and pointer networks

Abstract

This invention provides a Chinese event extraction method and system based on character and word features and pointer networks. The method includes the following steps: inputting the original Chinese sequence into a BERT model, segmenting and encoding it into high-dimensional vectors for character segmentation, and using the jieba word segmentation tool to segment the text; inputting the character-level vectors and word-level vectors into a BiLSTM network for character and word feature extraction, and fusing character and word features through an attention mechanism; and using a dual-sequence pointer network to identify the starting positions of trigger words and arguments to extract events from the text. The Chinese event extraction method of this invention addresses the problems of insufficient semantic representation, overlapping roles, and nested arguments in Chinese event extraction, improving the accuracy and generalization ability of the event extraction model.

Description

Technical Field

[0001] This invention relates to the field of event extraction, and more specifically, to a Chinese event extraction method and system based on word features and pointer networks. Background Technology

[0002] Event extraction is one of the most challenging tasks in the field of information extraction. It refers to extracting events of interest to users from unstructured information and presenting them to users in a structured manner. Event extraction has wide applications in event graph construction, public opinion analysis, information retrieval, and question answering, and has driven significant breakthroughs in the development of natural language processing technology. It is one of the powerful technical supports for the engineering applications of natural language processing.

[0003] Unlike English event element extraction, Chinese text is characterized by semantic ambiguity, complex word boundaries, and high-dimensional sparsity, making many English-based event extraction algorithms unsuitable for Chinese event extraction tasks. Existing Chinese event extraction methods suffer from insufficient semantic representation and inadequate modeling of trigger words and argument dependencies when argument roles overlap, leading to poor argument extraction performance. These issues are significant reasons for the difficulty in event extraction, and how to effectively utilize text features to improve the generalization and robustness of event extraction models remains a problem to be solved.

[0004] In view of this, the present invention is hereby proposed. Summary of the Invention

[0005] In view of this, this invention proposes a Chinese event extraction method and system based on character and word features and a pointer network. It constructs character vectors using the BERT model and simultaneously designs a word-level feature extraction network. Furthermore, it utilizes BiLSTM to extract character-level and word-level features separately, fusing the character vectors with weighted word vectors to obtain a more comprehensive and accurate semantically represented fused vector. This fused vector is input into a pointer network, where dual-sequence pointers are used to identify the starting positions of trigger words and arguments, making decoding more flexible and effectively solving the problems of role overlap and argument nesting, thus completing the final event extraction task. The aim is to address the problems of insufficient semantic representation, role overlap, and argument nesting in Chinese event extraction, and to improve the accuracy and generalization ability of the event extraction model.

[0006] Specifically, the present invention is achieved through the following technical solutions:

[0007] This invention provides a Chinese event extraction method based on word features and pointer networks, comprising the following steps:

[0008] The original Chinese sequence is input into the BERT model, segmented and encoded into high-dimensional vectors to achieve character segmentation, and the jieba word segmentation tool is used to segment the text.

[0009] Character-level vectors and word-level vectors are respectively input into a BiLSTM network for word feature extraction, and word features are fused through an attention mechanism.

[0010] By using a dual-sequence pointer network to identify the starting positions of trigger words and arguments respectively, extraction can be achieved from the text.

[0011] In addition to providing a Chinese event extraction method based on word features and pointer networks, this invention also provides a Chinese event extraction system based on word features and pointer networks, comprising:

[0012] The character segmentation and word segmentation module is used to input the original Chinese sequence into the BERT model, segment and encode it into a high-dimensional vector to achieve character segmentation, and use the jieba word segmentation tool to perform word segmentation on the text;

[0013] Word feature extraction and feature fusion module: Input character-level vectors and word-level vectors into the BiLSTM network to extract word features, and fuse word features through an attention mechanism;

[0014] Trigger word and argument extraction module: This module uses a dual-sequence pointer network to identify the starting positions of trigger words and arguments, enabling their extraction from the text.

[0015] In summary, the solution of the present invention has the following beneficial effects:

[0016] (1) When processing Chinese, the BERT model segments the text by character, but its deep learning characteristics enable it to capture the semantic relationship between characters and generate character vectors rich in semantic information. Therefore, the BERT model is used to segment and encode the input text, converting each character into a high-dimensional vector.

[0017] (2) In order to make up for the vocabulary information that BERT may lose when segmenting by character, the text is segmented by using the jieba word segmentation tool, and the word embedding vectors are obtained through pre-training, and optimized by model training after random initialization.

[0018] (3) In order to effectively capture the contextual information of the sequence, this paper inputs the character-level vector and word-level vector processed by the embedding layer into BiLSTM respectively, and uses BiLSTM to learn the deep-level contextual features in the input text. Through the attention mechanism, the character-level representation and word-level representation of the sequence are organically fused to generate a multi-granular hybrid representation of the sequence, thereby making full use of the character-level semantic information of BERT and the word-level information of jieba segmentation to obtain a more comprehensive and accurate semantic representation.

[0019] (4) To address the issues of overlapping roles and nested arguments in Chinese event extraction, a pointer network is used to construct the decoding module of the model. Dual-sequence pointers are used to identify the starting positions of trigger words and arguments respectively. The trigger word is treated as a special argument and extracted from the event text along with other arguments. A binary classification network model is then used to classify the argument roles, thereby resolving the issues of overlapping roles and nested arguments in event extraction. Attached Figure Description

[0020] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0021] Figure 1 This is a general framework diagram of the Chinese event extraction method of the present invention;

[0022] Figure 2 This is a schematic diagram of character-level feature extraction according to the present invention;

[0023] Figure 3 This is a schematic diagram of word-level feature extraction according to the present invention;

[0024] Figure 4 This is a flowchart illustrating a computer device provided in an embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure; rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms. Unless the context clearly indicates otherwise, it should be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, depending on the context. For example, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0028] Example

[0029] like Figure 1 As shown, the Chinese event extraction method of the present invention provides a Chinese event extraction method based on word features and pointer networks, including the following steps:

[0030] The original Chinese sequence is input into the BERT model, segmented and encoded into high-dimensional vectors to achieve character segmentation, and the jieba word segmentation tool is used to segment the text.

[0031] Character-level vectors and word-level vectors are respectively input into a BiLSTM network for word feature extraction, and word features are fused through an attention mechanism.

[0032] By using a dual-sequence pointer network to identify the starting positions of trigger words and arguments respectively, extraction can be achieved from the text.

[0033] The above method is mainly implemented by three modules: a character and word segmentation module based on character granularity and word granularity, a character and word feature extraction and fusion module based on BiLSTM, and a trigger word and argument extraction module based on pointer network. Character vectors are constructed using a BERT pre-trained model, and a word-level feature extraction network is designed simultaneously. BiLSTM is used to extract character-level and word-level features respectively. The character vectors are fused with weighted word vectors to obtain a more comprehensive and accurate semantic feature fusion vector. This fusion vector is input into the pointer network, where dual-sequence pointers are used to identify the starting positions of trigger words and arguments, making decoding more flexible and effectively solving the problems of role overlap and argument nesting, thus completing the final event extraction task.

[0034] The specific implementation of the character and word segmentation modules based on character granularity and word granularity follows these steps:

[0035] When a text sequence is input into the BERT model, the hidden vectors output by the last layer of BERT's Transformer Encoder are used as feature representations of the characters. The BERT model segments and encodes the input text, converting each character into a high-dimensional vector. The input character vector W = [w1, w2, ..., w...] is obtained. n ], where w n These are vector representations of the nth character, capturing the contextual semantic information of the character. To compensate for the lexical information that BERT may lose during character-by-character segmentation, this paper also uses the jieba word segmentation tool to segment the text and constructs a vocabulary by counting word frequencies. Each word in the vocabulary is assigned a unique ID, including special symbols such as... <pad>and <unk>This is then mapped to the corresponding word embedding vectors. These word embedding vectors are obtained through pre-training, and optimized through model training after random initialization.

[0036] The word feature extraction and feature fusion module based on BiLSTM is implemented as follows:

[0037] To effectively capture the contextual information of the sequence, the character-level vector and word-level vector processed by the embedding layer are input into the BiLSTM, such as... Figure 2 and Figure 3 As shown, by learning deep contextual features from the input text and fusing them with features output from a bidirectional long short-term memory network, the model's classification performance is improved. Where r... i Representing the input text embedding, BiLSTM uses a forward LSTM network to compute the state of the left part of the i-th character. Simultaneously, a reverse LSTM network is used to calculate the state of the right part of the i-th character. Then h i that is and The latent vector sequence obtained by concatenating two latent vectors. H represents the result of merging the latent vector sequences from 1 to i, which is the output of BiLSTM. The calculation process of the BiLSTM feature extraction layer can be formalized as follows:

[0038]

[0039]

[0040]

[0041] H = [h1, h2, ..., h n ]

[0042] To fully utilize the character-level semantic information of BERT and the word-level information of jieba segmentation, the character vectors generated by BERT are fused with the word embedding vectors corresponding to the segmentation results. For each character in the input text, its corresponding word is identified, and the word embedding vector of that word is weighted using an attention mechanism to reflect the importance of different words in the sentence. Assume the vector after word embedding encoding is V = [v...]. 1 ,v 2 ,…,v n Attention weights are calculated using the following formula:

[0043] [a 1 ,a 2 ,…,a n = softmax(linear(V))

[0044] Calculate the importance of each word, multiply the vector of each word by its corresponding weight, and then sum them. This can be expressed by the following formula:

[0045]

[0046] The weighted word vector V is obtained attn The weighted word embedding vector is concatenated with the character vector to obtain the fused word vector representation, denoted as H = [h1, h2, ..., h...]. n ], where h n This is the fusion vector of the nth character, which includes both the character's contextual information and the semantic information of the vocabulary. In this way, the embedding layer not only captures the character-level semantics of the text, but also enhances the understanding of complex semantics by introducing vocabulary information, providing a more comprehensive input for subsequent feature extraction and event extraction.

[0047] The trigger word and argument extraction module based on pointer networks is implemented according to the following steps:

[0048] Chinese event extraction tasks suffer from significant role overlap and argument nesting, posing a major challenge to event extraction methods. Confusion easily occurs during the annotation and prediction stages, affecting the final recognition results. Pointer networks, a sequence-to-sequence neural network model, solve the problem of selecting specific elements from an input sequence and are widely used in various natural language processing tasks.

[0049] The pointer network consists of an encoder, a decoder, and an attention mechanism. The encoder is usually a BiLSTM, which transforms the input sequence into a hidden state vector, where each hidden state represents a feature of each element in the input sequence. The decoder is usually an LSTM, which generates a probability distribution pointing to a certain position in the input sequence at each step.

[0050] Here, a dual-sequence pointer approach is used to locate the start and end positions of trigger words and arguments in the input sequence, and the trigger word is treated as a special argument, extracted from the text along with the arguments. Specifically, the annotations in the original dataset are defined as follows:

[0051] L=(E,Ti_s,Ti_e,R,Ar_s,Ar_e)

[0052] Where L represents an end-to-end tag, E represents the event type, Ti_s and Ti_e represent the start and end positions of the trigger word, respectively, R represents the argument role, and Ar_s and Ar_e represent the start and end positions of the argument, respectively. A single event text often contains multiple arguments and corresponds to multiple event types; therefore, a single event text may contain multiple end-to-end tags L, which are defined as follows:

[0053]

[0054] Where C represents the complete set of tags corresponding to a single event text.

[0055] This module designs two pointers to locate the first and last positions of trigger words and arguments in the event text sequence, respectively, and then uses a binary classification network model to classify the argument roles. This module uses a pointer network with multi-round decoding, cascaded pointers, and an autoregressive output structure to flexibly handle the nesting and hierarchical relationships between events, accurately capture nested events and roles, and effectively solve the problems of role overlap and argument nesting in event extraction.

[0056] In addition to providing a Chinese event extraction method, this invention also provides a Chinese event extraction system, specifically including:

[0057] The character segmentation and word segmentation module is used to input the original Chinese sequence into the BERT model, segment and encode it into a high-dimensional vector to achieve character segmentation, and use the jieba word segmentation tool to perform word segmentation on the text;

[0058] Word feature extraction and feature fusion module: Input character-level vectors and word-level vectors into the BiLSTM network to extract word features, and fuse word features through an attention mechanism;

[0059] Trigger word and argument extraction module: This module uses a dual-sequence pointer network to identify the starting positions of trigger words and arguments, thereby extracting them from the text.

[0060] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method implementation examples, which will not be repeated here.

[0061] The Chinese event extraction method of the present invention has the following beneficial effects:

[0062] This study addresses issues such as insufficient semantic representation, overlapping roles, and nested arguments in Chinese event extraction, improving the accuracy and generalization ability of the event extraction model. It constructs character vectors using a BERT pre-trained model and simultaneously designs a word-level feature extraction network. BiLSTM is then used to extract character-level and word-level features separately. The character vectors are fused with weighted word vectors to obtain a more comprehensive and accurate fused vector. This fused vector is input into a pointer network, where dual-sequence pointers identify the starting positions of trigger words and arguments, making decoding more flexible and effectively resolving issues of overlapping roles and nested arguments, thus completing the final event extraction task.

[0063] Figure 4 This is a schematic diagram of the structure of a computer device disclosed in this invention. (Reference) Figure 4 As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403 and is used to store computer instructions executable by the processor 401. The processor 401 is used to read computer instructions from the memory 402 to implement the steps of the method described in any of the above embodiments.

[0064] For the above-described apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0065] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0066] Finally, it should be noted that although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe the features of specific embodiments of a particular invention. Certain features described in the various embodiments of this specification may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0067] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0068] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0069] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.< / unk> < / pad>

Claims

1. A Chinese event extraction method based on word features and pointer networks, characterized in that, Comprising the following steps: The original Chinese sequence is input into the BERT model, segmented and encoded into a high-dimensional vector to realize the word-by-word processing, and the jieba word segmentation tool is used for word segmentation processing of the text; The character-level vector and the word-level vector are input into the BiLSTM network respectively to extract word features, and the attention mechanism is used for word feature fusion; The starting positions of the trigger words and the arguments are identified by using the double sequence pointer network to realize extraction from the text; The fused method comprises the following steps: calculating the state of the left part of the i-th character by using a forward LSTM network Meanwhile, a reverse LSTM network is used to calculate the state of the right part of the i-th character h i is and The hidden vector sequence after splicing of the two hidden vectors, H represents the combined result of the hidden vector sequence from 1 to i, that is, the output result of the BiLSTM, and the specific formula is as follows: H = [h1, h2,..., h n ] where r i represents an input text embedding; Suppose the vector after word embedding encoding is V = [v 1 ,v 2 ,…,v n ], the attention weight is calculated by the following formula: [a 1 ,a 2 ,…,a n ] = softmax(line max(V)) The importance of each word is calculated, and the vector of each word is multiplied by the corresponding weight and added, which can be represented by the following formula: obtaining the weighted word vector V attn ; The weighted word embedding vector is concatenated with the character vector to obtain the fused word vector representation, denoted as H = [h1, h2, ..., h n ], where h n It is the fusion vector of the nth character; The double sequence pointer network is composed of an encoder, a decoder and an attention mechanism, the encoder is BiLSTM, and the input sequence is converted into a hidden state vector; the decoder is LSTM, and a probability distribution pointing to a position in the input sequence is generated at each step; The method for identifying the starting positions of the trigger words and the arguments by using the double sequence pointer network to realize extraction from the text comprises the following steps: The annotations in the original data set are as follows: L=(E,Ti_s,Ti_e,R,Ar_s,Ar_e) Wherein, L represents an end-to-end label, E represents an event type, Ti_s and Ti_e represent the starting position and the ending position of the trigger word respectively, R represents the argument role, Ar_s and Ar_e represent the starting position and the ending position of the argument respectively, and the definitions are as follows: Wherein, C is the total label set corresponding to a single event text.

2. The Chinese event extraction method of claim 1, wherein, The method of character-by-character processing comprises the following steps: using the hidden vectors output by the last layer of the BERT model as the feature representation of the characters, splitting and encoding the input text, converting each character into a high-dimensional vector, obtaining the input character vector W=[w1, w2, …, w n ], wherein w n is the vector representation of the nth character.

3. The Chinese event extraction method of claim 1, wherein, The method of performing word segmentation includes the following steps: counting word frequency to construct a word list, each word in the word list is assigned a unique ID, including special symbols such as <pad>and <unk>And mapped to the corresponding word embedding vector.< / unk> < / pad> 4. A Chinese event extraction system based on word features and pointer networks, for implementing the Chinese event extraction method of any one of claims 1-3, characterized in that, Comprise: Word-by-word word segmentation module: used for inputting the original Chinese sequence into the BERT model, segmenting and encoding into a high-dimensional vector to realize word-by-word processing, and using the jieba word segmentation tool for word segmentation processing of the text; Word feature extraction and feature fusion module: the character-level vector and the word-level vector are input into the BiLSTM network respectively to extract word features, and the attention mechanism is used for word feature fusion; Trigger word and argument extraction module: used for identifying the starting positions of the trigger words and the arguments by using the double sequence pointer network to realize extraction from the text.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by the processor to execute the steps of the Chinese event extraction method in any one of claims 1-3.

6. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the Chinese event extraction method in any one of claims 1-3.

Images