A document-level Chinese event extraction method

Abstract

A document-level Chinese event extraction method, comprising the following steps: 1) screening Chinese event data from a document, and obtaining an event data set after preprocessing the obtained Chinese event data; 2) labeling typical events in different fields for the event data set to obtain a labeled data set of each event category; and 3) constructing a Chinese general field event extraction model: based on a convolutional neural network and a graph convolutional neural network, performing feature extraction on the labeled data set, and then performing event category classification and argument role extraction. The method reconstructs the original document into a "image" structure and a graph structure, completes the event classification and corresponding argument classification tasks, and improves the event extraction precision and the generalization ability of the model.

Description

Technical Field

[0001] The present invention relates to event data processing technology, in particular to a document-level Chinese event extraction method. Background Art

[0002] With the rapid development of information technology and media such as news media and social media, information meets people's daily information acquisition needs in various forms, such as image form, text form and streaming media form. The wide variety and huge amount of information groups have caused problems such as information redundancy and poor readability. How to automatically extract important structured information from unstructured information has become an urgent problem to be solved. Events are the basic units of information representation. How to quickly and effectively extract events is crucial for information extraction and information understanding tasks. In order to respond to the call of "Internet +", various fields are becoming increasingly closely connected with computers, such as finance and medicine.

[0003] Currently, there are two main types of event extraction models: pipeline models and joint models. The pipeline model refers to performing the various subtasks in event extraction (event detection and classification, argument detection and classification) in a "serial" manner. This method is relatively simple to implement and easy to operate, and the completion of the previous task will reduce the number of detection samples for the next task. However, if an error occurs in the previous task, it will affect the detection accuracy of the next task, thereby causing the "error propagation" problem. The joint model refers to the ability to combine the connections and dependencies between the various subtasks in event extraction and to perform them in a "parallel" manner. This method can minimize the "error propagation" problem brought about by the pipeline model during model training, but this problem still exists during model testing.

[0004] In news websites, text generally appears in the form of documents, so sentence-level event extraction methods are not applicable to this task. In document-level event extraction tasks, since the text span of a document from beginning to end is large, and the arguments of an event may be distributed in multiple sentences, how to solve the argument dispersion problem is the most important and urgent problem that needs to be solved in document-level event extraction tasks. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a document-level Chinese event extraction method, which adopts a Chinese document field event extraction method based on convolutional neural networks and graph convolutional neural networks to reconstruct the original document into an "image" structure and a graph structure, complete event classification and corresponding argument classification tasks, and improve event extraction accuracy and model generalization ability.

[0006] To achieve the above technical objectives, the adopted technical solution is: a document-level Chinese event extraction method, the method comprising:

[0007] Step 1, coarse screening of Chinese event data from documents, obtaining an event data set after preprocessing the obtained Chinese event data;

[0008] Step 2, labeling typical events under different fields for the event data set to obtain a labeled data set for each event category;

[0009] Step 3, constructing a Chinese general field event extraction model: based on convolutional neural network and graph convolutional neural network, extracting features from the labeled data set, and then classifying event categories and argument role extraction.

[0010] The method for coarse screening of Chinese event data from documents is to set a general field event keyword dictionary and set a regular expression to filter irrelevant events.

[0011] The method for preprocessing Chinese event data includes filtering irrelevant events in the original data and deleting special tags in the website.

[0012] The Chinese general field event extraction model is constructed, specifically including:

[0013] Step 3.1: The labeled data set is divided into sentences as the granularity, and then all the divided sentences are input into the pre-training model in turn to obtain the corresponding sentence vector e s .

[0014] Step 3.2: The sentence is divided into entities as the granularity, and then all the divided entities are input into the pre-training model in turn to obtain the corresponding entity vector, and then the entity vector is input into the Maxpool layer of the convolutional neural network to compress the features to obtain the compressed entity vector e t .

[0015] Step 3.3: For the compressed entity vector of step 3.2, arrange it into a document vector in the order from top to bottom, so that the row vector and the vertical vector of the arranged document vector have equal dimensions. Copy the document vector with equal dimensions of row vector and vertical vector twice to obtain the final document vector D i .

[0016] Step 3.4: For the sentence vector e s in step 3.1 and the compressed entity vector e t in step 3.2, construct a document graph structure vector D g according to the dependency relationship between entities in different sentences and entities in the same sentence.

[0017] Step 3.5: For the document vector D i in step 3.3 and the document graph structure vector D g, using convolutional neural networks and graph convolutional neural networks for feature extraction respectively;

[0018] Step 3.6: Based on the event categories and arguments of typical events during the annotation process, synthesize the document vector D extracted in step 3.5 i and the document graph structure vector D in step 3.4 g features, classify the event type and extract the argument role for each sentence.

[0019] The specific method for judging the event type and extracting argument roles for each sentence is:

[0020] Step 3.6.1: Transform the document graph structure vector D g The feature and document vector D i The features are spliced ​​and then filtered. Finally, the filtered feature vector is input into the fully connected layer of the convolutional neural network to obtain the corresponding typical event category probability. The maximum probability is taken as the corresponding event category to complete the event category classification;

[0021] Step 3.6.2: Document graph structure vector D g The features extracted by the graph convolutional neural network are then input into the Maxpool layer of the convolutional neural network to obtain candidate argument vectors. Finally, the candidate argument vectors are input into the fully connected layer of the convolutional neural network to obtain the argument probability of the corresponding event category. The maximum probability is taken as the corresponding argument role to complete the argument role extraction.

[0022] The pre-training model can use the BERT model, Word2Vec model or ELMO model.

[0023] The specific method for making the row vector and vertical vector dimensions of the arranged document vector equal is: if the number of compressed entity vectors is less than the row dimension of the compressed entity vector, then add a 0 vector of the same dimension; if the number of compressed entity vectors is greater than the row dimension of the compressed entity vector, then truncate the compressed entity vector.

[0024] The method for feature screening in step 3.6.1 is to use the Transformer model, long short-term memory network and recurrent neural network.

[0025] The beneficial effects of the present invention are:

[0026] 1. The document-level event extraction method proposed in this invention takes documents as units and reassembles a document into an "image" so that the "image" contains all the information of the document. It also uses a convolutional neural network to extract features from the "image" and finally compresses the entire document information into a one-dimensional vector, which can obtain the semantic features of the entire document.

[0027] 2. The document-level event extraction method proposed in the present invention reconstructs the document into a graph structure through the dependency relationship between entities in the sentences in the document, so that even if different arguments of the same event are distributed in different sentences, the connections and dependencies between them can be increased through the above graph structure. In addition, in the process of event classification and argument classification, the local features and global features of the document are combined to enhance feature utilization and improve extraction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Flowchart of the present invention;

[0029] Figure 2 This is a structural diagram of the model of the present invention. DETAILED DESCRIPTION

[0030] This implementation scheme is a document-level event extraction method for the general Chinese field. The sentence-level event extraction method cannot solve the argument dispersion problem in the document-level event extraction task. By reconstructing the original document into an "image" form and graph structure, the global features of the document and the local features of a single sentence are fully utilized to improve the ability of document-level event extraction.

[0031] Combine Figure 1 and Figure 2 The present invention relates to a document-level Chinese event extraction method, which specifically includes the following steps:

[0032] Step 1: We coarsely screened Chinese event data from various news websites and publicly available general domain event extraction datasets. We then set up a general domain event key dictionary and used regular expressions to filter out irrelevant events. Since some of the raw data came from general domain news websites, HTML tags were present in the data, so we also needed to set up regular expressions to remove them. We then preprocessed the acquired spoken event data, including filtering out irrelevant events and removing special tags from the websites. This ultimately resulted in the event dataset.

[0033] Step 2: Based on the characteristics of the event dataset, typical events in different fields are formulated, such as conflict, death, marriage, meeting, transaction, etc. The arguments of each typical event include key elements such as people, place and time, and then manually labeled to obtain a labeled dataset for each type of event.

[0034] Step 3: Build a Chinese general domain event extraction model: Based on convolutional neural networks and graph convolutional neural networks, after extracting features from the annotated dataset, perform event classification and argument role classification.

[0035] Step 3.1: Split the labeled dataset into d=[s1,s2,...,sn ], where s i Indicates the i-th sentence, n indicates the number of sentences in a document. In the actual segmentation process, since the present invention is aimed at Chinese text, it can be segmented according to special punctuation marks such as period, semicolon, etc., and then s i Input into the pre-training model in sequence, Figure 2 Taking the BERT model as an example, the encoding of the last layer of the BERT model output is obtained as the corresponding sentence vector e s , the dimension is 1×m, where m is the row dimension of the sentence vector. When using the BERT model, m is 768.

[0036] Pre-trained models can use the BERT model, the Word2Vec model, or the ELMO model. When using the Word2Vec model, word embeddings can be of any dimension; when using the ELMO model, word embeddings have a dimension of 512. The following detailed description uses the BERT model.

[0037] Step 3.2: Segment the sentence into s=[t1,t2,...,t q ], where t i Indicates the i-th entity, q indicates the number of entities in a sentence. In the actual segmentation process, the LAC word segmentation tool is used for entity segmentation. The LAC word segmentation tool is trained based on Chinese entities. Therefore, the result after word segmentation can ensure the accuracy of entity edge segmentation as much as possible. For example, the word segmentation result of "Shanghai Wanye Enterprise Co., Ltd.'s announcement on the increase of company executives' holdings of company stocks" is "'Shanghai Wanye Enterprise Co., Ltd.'; 'about'; 'company'; 'executives'; 'increase holdings'; 'company'; 'stock'; 'of'; 'announcement'". Then t i The BERT pre-trained model is input in sequence, and the encoding of the last layer of the BERT model output is obtained as the corresponding entity vector with a dimension of 1×768. The entity vector is then input into the Maxpool layer of the convolutional neural network to further compress the features. The convolution kernel of the Maxpool layer is 3×3 and the step size is 2. The compressed entity vector e is obtained. t , with dimensions of 1×384.

[0038] Step 3.3: For the entity vector e in step 4 t , arrange them in order from top to bottom. If the number of entity vectors is less than the row dimension of the entity vector, add a 0 vector of the same dimension. If the number of entity vectors is greater than the row dimension of the entity vector, truncate the entity vector to ensure that the row vector and vertical vector dimensions of the arranged document vector are equal. Finally, copy the obtained document vector twice to obtain the final document vector D i , whose dimension is (3, 384, 384).

[0039] Step 3.4: For the sentence vector e in step 3 s and the entity vector e in step 4 t , construct the document graph structure vector D according to the co-occurrence of entities in different sentences g There are two types of nodes in the graph structure, namely sentence nodes and argument nodes, and four types of edges, namely sentence nodes and sentence nodes; sentence nodes and argument nodes; argument nodes and argument nodes in the same sentence and argument nodes and argument nodes between different sentences. Its dimensions change with the changes in the entities and edges in the document.

[0040] Step 3.5: For the document vector D in step 5 i and the document graph structure vector D in step 6 g Convolutional neural network (ResNet18) and graph convolutional neural network (3 layers) are used for feature extraction respectively. Convolutional neural network can be regarded as a feature extractor of image data. In the present invention, ResNet18 is used as a convolutional neural network feature extractor. ResNet18 has a total of 18 layers. The input image size of its network structure is 224×224 pixels and the number of channels is 3. After the first layer of convolution, the image size is reduced to 112×112 and the number of channels is increased to 64. After the maximum pooling layer, the image is further reduced to 56×56, and the number of channels does not increase. After the first two steps, it enters the residual part. After each part of the residual image size is reduced to half of its original size, and the number of channels is increased to 2 times the original size. Image dimensionality reduction is achieved through a convolutional layer with a step size of 2. After 4 residual operations, the final image size is reduced to 7×7 and the number of channels is increased to 512. Finally, the average pooling layer and the fully connected layer are connected.

[0041] Graph convolutional neural networks have similar functions to convolutional neural networks. They both function as feature extractors, but they target graph data. For the central node i and the number of layers l, graph convolutional neural networks can be defined as:

[0042]

[0043] in, represents the feature vector of node i in layer l, represents the feature vector of node i in the l+1 layer, c ij is the normalization factor, N i Represents the neighbors of node i, including itself, R i represents the type of node i, is a learnable parameter. In this invention, the number of layers of the graph convolutional neural network is set to 3. Step 3.6: In the document graph structure vector D g In the example, the feature vector dimension of each sentence is 1×768, and the document vector Di After feature extraction using a convolutional neural network, the dimension is 1×768. The concatenation of the two features reduces the dimension to 1×1536. Feature filtering then generates a vector with a dimension of 1×768. Finally, this vector is fed into a fully connected layer to obtain the probabilities of typical event categories. The maximum probability is taken as the corresponding event category, completing event classification. The feature filtering method in step 7 utilizes a Transformer model, a long short-term memory network, and a recurrent neural network. Figure 2 The Transformer model is used for feature screening.

[0044] Step 3.7: The corresponding typical event category is obtained from step 3.6, and then the document graph structure vector D is converted to g The features extracted by the graph convolutional neural network are input into the Maxpool layer, and the feature dimensions are uniformly adjusted to n×1×768, where n is the number of candidate arguments. Finally, the candidate argument vectors are input into the fully connected layer to obtain the argument probabilities corresponding to typical event categories. The maximum probability is taken as the corresponding argument role to complete the argument role classification.

Claims

1. A document-level Chinese event extraction method, characterized in that: The method comprises: Step 1: roughly screen Chinese event data from the document, pre-process the acquired Chinese event data and obtain an event dataset; Step 2: Label typical events in different fields for the event dataset to obtain a labeled dataset for each event category; Step 3: Build a Chinese general domain event extraction model: Based on convolutional neural networks and graph convolutional neural networks, after extracting features from the annotated dataset, perform event category classification and argument role extraction; Construct a Chinese general domain event extraction model, specifically including: Step 3.1: Segment the labeled dataset into sentences, and then input all the segmented sentences into the pre-trained model to obtain the corresponding sentence vectors. ; Step 3.2: Segment the sentence into entities, then input all the segmented entities into the pre-trained model in turn to obtain the corresponding entity vectors, and then input the entity vectors into the Maxpool layer of the convolutional neural network to compress the features and obtain the compressed entity vectors. ; Step 3.3: Arrange the compressed entity vectors from step 3.2 into document vectors in order from top to bottom, so that the row vector and vertical vector dimensions of the arranged document vectors are equal. Copy the document vector with equal row vector and vertical vector dimensions twice to obtain the final three-dimensional document vector. ; Step 3.4: For the sentence vector in step 3.1 and the compressed entity vector in step 3.2 , construct document graph structure vector based on the dependency between entities in different sentences and between entities in the same sentence ; Step 3.5: For the document vector in step 3.3 and the document graph structure vector in step 3.4 , using convolutional neural networks and graph convolutional neural networks for feature extraction respectively; Step 3.6: Based on the event categories and arguments of typical events during the annotation process, synthesize the document vectors extracted in step 3.5 and the document graph structure vector in step 3.4 The event category is classified and the argument role is extracted for each sentence based on the features.

2. The document-level Chinese event extraction method according to claim 1, wherein: The method for roughly filtering Chinese event data from documents is to set up a common domain event key dictionary and set up a regular expression to filter out irrelevant events.

3. The document-level Chinese event extraction method according to claim 1, wherein: Methods for preprocessing Chinese event data include filtering out irrelevant events in the original data and deleting special tags in the website.

4. The document-level Chinese event extraction method according to claim 1, wherein: The specific method for judging the event type and extracting argument roles for each sentence is: Step 3.6.

1. Vectorize the document graph structure The feature and document vectors The features are spliced ​​and then filtered. Finally, the filtered feature vector is input into the fully connected layer of the convolutional neural network to obtain the corresponding typical event category probability. The maximum probability is taken as the corresponding event category to complete the event category classification; Step 3.6.2: Vectorize the document graph structure The features extracted by the graph convolutional neural network are then input into the Maxpool layer of the convolutional neural network to obtain candidate argument vectors. Finally, the candidate argument vectors are input into the fully connected layer of the convolutional neural network to obtain the argument probability of the corresponding event category. The maximum probability is taken as the corresponding argument role to complete the argument role extraction.

5. The document-level Chinese event extraction method according to claim 1, wherein: The pre-training model uses the BERT model, Word2Vec model, or ELMO model.

6. The document-level Chinese event extraction method according to claim 1, wherein: The specific method for making the row vector and vertical vector dimensions of the arranged document vector equal is: if the number of compressed entity vectors is less than the row dimension of the compressed entity vector, then add a 0 vector of the same dimension; if the number of compressed entity vectors is greater than the row dimension of the compressed entity vector, then truncate the compressed entity vector.

7. The document-level Chinese event extraction method according to claim 4, wherein: The method for feature screening in step 3.6.1 is to use the Transformer model, long short-term memory network and recurrent neural network.

Images