A chinese domain event extraction method

Abstract

The application discloses a Chinese domain event extraction method, comprising the following steps: selecting original domain corpus from a domain database, constructing a domain data set and a professional word table, and training a militaryBERT model; adopting the militaryBERT model, a dependency syntax analysis tool DDParser and a part-of-speech analysis tool Jieba to perform data preprocessing on input corpus of an event extraction data set, and obtaining final original feature representation. The Chinese domain event extraction method adopts the militaryBERT model, syntax dependency analysis and part-of-speech analysis to perform data preprocessing on input corpus of the event extraction data set, and obtains original feature representation, thereby enhancing the context semantic representation capability of an event extraction model; the Chinese domain event extraction method constructs an event extraction model comprising a first feature extraction network and a second feature extraction network, and effectively improves the accuracy of event extraction by fusing features extracted by each feature extraction network.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of natural language processing and knowledge graph, and particularly relates to a Chinese domain event extraction method. BACKGROUND

[0002] The event extraction task mainly includes four types of event type identification, trigger word identification, argument and argument role identification, and the domain event extraction method mentioned in the application is mainly applicable to the event type identification and event trigger word detection tasks.

[0003] The event extraction method mainly includes three types: the rule-based method relies on domain experts to develop event extraction rules, which has high accuracy in specific fields, but consumes more manpower and has poor portability. The machine learning-based method models the event extraction task as a multi-classification problem, trains multiple different models through a large amount of corpus, and has the problems of difficult data set acquisition and error propagation. The deep learning-based method mostly models the event extraction as an end-to-end model, uses word vectors containing rich features as input, and automatically extracts deep features required by the task using the model, effectively avoiding complex manual work.

[0004] The existing deep learning-based event extraction method defaults that the labels on the training set and the validation set are correct, and mostly achieves good results on English data sets, partly because English is a minimum ideographic unit with words, and different words are separated by spaces. When migrating the above model to the Chinese scene, the Chinese text input needs to be segmented first, and the segmentation effect directly affects the accuracy of the subsequent model event extraction. The neural network model used includes convolutional neural network, recurrent neural network, etc. However, in the actual scene, the data set will have annotation errors, and the existing technology does not consider this point, so the method and technology used need to be improved. SUMMARY

[0005] The purpose of the application is to solve the problems raised in the background art, and to provide a Chinese domain event extraction method.

[0006] To achieve the above purpose, the technical scheme adopted by the application is:

[0007] The Chinese domain event extraction method provided by the application comprises selecting original domain corpus from a domain database, constructing a domain data set and a professional word table, and training a militaryBERT model.

[0008] The input corpus of the event extraction data set is preprocessed by the militaryBERT model, the dependency syntax analysis tool DDParser and the part-of-speech analysis tool Jieba to obtain the final original feature representation.

[0009] The event extraction model including two different feature extraction networks is constructed, the original feature representation and the input corpus are sequentially input into the first feature extraction network and the second feature extraction network, the corresponding loss weight is adjusted according to the number of each event category, and the event extraction model is trained by using a gradient error back propagation algorithm.

[0010] A new classification threshold is set, the input corpus is input into the trained event extraction model, and the event type is predicted based on the conditional probability.

[0011] Preferably, the input corpus of the event extraction data set is subjected to data preprocessing by using the militaryBERT model to obtain the final original feature representation, including:

[0012] The input corpus of the event extraction data set is subjected to entity recognition by using the militaryBERT model, a mark symbol is added before and after the corresponding entity of the input corpus according to the recognition result, the added mark symbol and the recognized entity type are one-to-one corresponding, the current character sequence is encoded by using the militaryBERT model to obtain a character-level character encoding vector.

[0013] The dependency syntax analysis tree between the syntax units is constructed by using the dependency syntax analysis tool DDParser, each character string sequence id of the current dependent word is pointed to the governing word, the obtained dependency label information is encoded to obtain a character-level dependency label encoding vector.

[0014] The input corpus is subjected to part-of-speech analysis by using the part-of-speech analysis tool Jieba, the part-of-speech label of the current word is assigned to each character constituting the current word, the obtained part-of-speech label information is encoded to obtain a character-level part-of-speech label encoding vector.

[0015] The character-level character encoding vector, the character-level dependency label encoding vector and the character-level part-of-speech label encoding vector are spliced according to the character sequence to obtain the final original feature representation.

[0016] Preferably, the event extraction model including two different feature extraction networks is constructed, the original feature representation and the input corpus are sequentially input into the first feature extraction network and the second feature extraction network, including:

[0017] The first feature extraction network is designed as follows:

[0018] The first layer is a bidirectional LSTM layer, the original feature representation is input, and the timing information of the context of the target word is captured;

[0019] The second layer is a first full connection layer, which is used to integrate and extract the text deep features, and convert the deep features into a regular matrix;

[0020] The third layer is a multi-layer graph neural network, the edge of the graph neural network is updated by updating the weight matrix between each character in the dependency syntax analysis tree, and the number of graph neural networks used in the current layer is adjusted by the following formula:

[0021]

[0022] Wherein, A represents the adjacency matrix; A ij represents that there is a single edge before node i to node j, W l represents the weight matrix of the lth layer of graph neural network, b l represents the bias matrix of the lth layer of graph neural network, represents the hidden layer state of the (l-1)th layer of graph neural network about j node, represents the hidden layer state of the lth layer of graph neural network about j node, and sigma represents the ReLU activation function, and pooling represents the pooling operation.

[0023] The fourth layer is a second full connection layer, which is used to compress the output dimension of the multi-layer graph neural network to the size of the type number plus one.

[0024] The design of the second feature extraction network is as follows:

[0025] The first layer is a militaryBERT model, the input corpus of the event extraction dataset is input, the multi-head self-attention mechanism is used to capture the semantic features of the word level, and the high-dimensional dynamic word vector is converted;

[0026] The second layer is a CRF layer, which learns the relationship between labels to improve the prediction accuracy.

[0027] The output of the first feature extraction network and the output of the second feature extraction network are spliced.

[0028] And when training the event extraction model, the gradient error back propagation algorithm is used to jointly train the first feature extraction network and the second feature extraction network.

[0029] Preferably, the loss weight of each event category is adjusted according to the number of each event category, including:

[0030] The overall loss function of the event extraction model is:

[0031] loss=αloss1+(1-α)loss2

[0032] Wherein, alpha represents a hyperparameter, loss1 represents the loss of the first feature extraction network, and loss2 represents the loss of the second feature extraction network.

[0033] The loss weight is preset according to the number of each event category:

[0034]

[0035] wherein, W p represents the loss weight of the current pth event, n p represents the number of the current pth event, and m represents the number of event categories.

[0036] and the loss of the first feature extraction network and the second feature extraction network is:

[0037]

[0038] wherein, W represents the loss weight of the event category to which the current input corpus belongs, X i represents the real event label of the current input corpus, represents the predicted event label of the current input corpus, and k represents the number of the current input corpus.

[0039] Preferably, a new classification threshold is set, the input corpus is input into the trained event extraction model, and the event type is predicted based on the conditional probability, including:

[0040] The output of the event extraction model is normalized by using a Softmax layer:

[0041]

[0042] wherein, z represents an input vector of the Softmax layer, z i represents the value corresponding to the ith position of the input vector, z q represents the value corresponding to the qth position of the input vector.

[0043] The classification threshold of each event category is modified:

[0044]

[0045] wherein, θ p represents the classification threshold of the current pth event, and p is an integer in m+1.

[0046] According to the obtained probability distribution, it is judged whether the predicted probability of the event type with the largest predicted probability is greater than the classification threshold of the current category event, if yes, the event extraction model predicts the output as the current category event, otherwise, the largest probability value in the current probability distribution is deleted, the remaining part is taken as a new probability distribution output, and the step is repeated until the predicted probability is greater than the classification threshold of the current category event.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] 1. The Chinese domain event extraction method adopts a militaryBERT model, syntactic dependency analysis and part-of-speech analysis to perform data preprocessing on input corpus of an event extraction dataset, to obtain original feature representation, and enhances the context semantic representation capability of the event extraction model;

[0049] 2. The Chinese domain event extraction method constructs an event extraction model including a first feature extraction network and a second feature extraction network, and effectively improves the accuracy of event extraction by fusing the features extracted by each feature extraction network. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The flowchart of the Chinese domain event extraction method of the present application is shown in the figure;

[0051] Figure 2 The structural diagram of the event extraction model of the present application is shown in the figure;

[0052] Figure 3 The schematic diagram of the dependency syntax analysis tree at the word level in the data preprocessing of the present application is shown in the figure;

[0053] Figure 4 The schematic diagram of the dependency syntax analysis tree at the character level in the data preprocessing of the present application is shown in the figure;

[0054] Figure 5 The structural diagram of the first feature extraction network of the present application is shown in the figure;

[0055] Figure 6 The structural diagram of the second feature extraction network of the present application is shown in the figure. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0057] It should be noted that when a component is referred to as being "connected" with another component, it can be directly connected with the other component or there can be a middle component. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

[0058] As shown in the figure, a Chinese domain event extraction method comprises: Figures 1-6

[0059] ​It should be noted that this Chinese domain event extraction method is applied to the military field.

[0060] Step S1: Select original domain corpora from the domain database, construct a domain dataset and a professional vocabulary, and train the militaryBERT model (effectively enriching the semantic features of domain texts).

[0061] Step S2: The input corpus of the event extraction dataset is preprocessed using the militaryBERT model, the dependency parsing tool DDParser, and the part-of-speech tagging tool Jieba to obtain the final raw feature representation.

[0062] Specifically, the militaryBERT model is used to perform entity recognition on the input corpus of the event extraction dataset. Based on the recognition results, markers are added before and after the corresponding entities in the input corpus, and the added markers correspond one-to-one with the recognized entity types. The militaryBERT model is then used to encode the current character sequence (which refers to the character sequence formed by adding markers before and after the corresponding entities in the input corpus based on the entity recognition results of the event extraction dataset) to obtain a character-level character encoding vector.

[0063] The dependency parsing tool DDParser is used to construct dependency parsing trees between syntactic units (this method builds character-level dependency parsing trees based on word-level sequential parsing trees). Each character of the current dependent word is linked to the string sequence ID of the parent word. The resulting dependency tag information is encoded to obtain a character-level dependency tag encoding vector. A schematic diagram of the word-level dependency parsing tree is shown below. Figure 3 As shown, the character-level dependency syntax tree is as follows: Figure 4 As shown ( Figure 4 (Only some of the character dependencies are shown in the text.)

[0064] The part-of-speech tagging tool Jieba is used to perform part-of-speech tagging on the input corpus (the word-level part-of-speech tagging is processed into character-level part-of-speech tagging). The part-of-speech tag of the current word is assigned to each character that makes up the current word. The obtained part-of-speech tag information is encoded to obtain a character-level part-of-speech tag encoding vector.

[0065] The character-level character encoding vector, character-level dependency tag encoding vector, and character-level part-of-speech tag encoding vector are concatenated according to the character sequence to obtain the final original feature representation.

[0066] Step S3: Construct an event extraction model that includes two different feature extraction networks. Input the original feature representation and the input corpus into the first feature extraction network and the second feature extraction network in sequence. Adjust the corresponding loss weights according to the number of each event category. Train the event extraction model using the gradient error backpropagation algorithm.

[0067] Specifically, the design of the first feature extraction network is as follows:

[0068] The first layer is a bidirectional LSTM layer, which takes the original feature representation as input, captures the temporal information of the target word context, and uses a Dropout layer to alleviate overfitting.

[0069] The second layer is the first fully connected layer, which is used to integrate the extracted deep text features and convert the deep features into a regular matrix;

[0070] The third layer is a multi-layer graph neural network. It updates the edges of the graph neural network by updating the weight matrix between each character in the dependency parsing tree. The number of graph neural networks used in the current layer is adjusted by the following formula:

[0071]

[0072] Where A represents the adjacency matrix; A ij W indicates that there is a one-way edge between node i and node j. l Let b represent the weight matrix of the l-th layer of the graph neural network. l This represents the bias matrix of the l-th layer of the graph neural network. This represents the hidden state of the (l-1)th layer graph neural network with respect to node j. Let σ represent the hidden state of the l-th layer graph neural network with respect to node j, σ represent the ReLU activation function, and pooling represent the pooling operation. It should be noted that all node features pass through multiple layers of graph neural networks. In order to reduce parameters, mean pooling is used after each layer of graph neural network. In order to prevent overfitting, activation functions and dropout layers are used.

[0073] The fourth layer is the second fully connected layer, which is used to compress the output dimension of the multi-layer graph neural network to the size of the number of event types plus one (if the number of event types in the event extraction dataset is n, then the required output dimension here is n+1).

[0074] The design of the second feature extraction network is as follows:

[0075] The first layer is the militaryBERT model, which takes the input corpus of the event extraction dataset as input, uses a multi-head self-attention mechanism to capture word-level semantic features, and converts them into high-dimensional dynamic word vectors;

[0076] The second layer is the CRF layer, which learns the relationships between labels and improves prediction accuracy.

[0077] Then the outputs of the first feature extraction network and the second feature extraction network are concatenated (the output dimensions of the first feature extraction network and the second feature extraction network are the same).

[0078] Adjust the corresponding loss weights based on the number of events in each category (effectively improving the problem of class imbalance in the dataset):

[0079] The overall loss function of the event extraction model is:

[0080] loss = αloss1 + (1-α)loss2

[0081] Where α represents the hyperparameter, loss1 represents the loss of the first feature extraction network, and loss2 represents the loss of the second feature extraction network.

[0082] Preset loss weights based on the number of events in each category:

[0083]

[0084] Among them, W p This represents the loss weight for the current p-th type of event, where n p This represents the number of events of type p, and m represents the number of event types.

[0085] And the losses of the first feature extraction network and the second feature extraction network are:

[0086]

[0087] Where W represents the loss weight of the event category to which the current input corpus belongs, and X... i This indicates the actual event labels in the current input corpus. This represents the event label predicted from the current input corpus, and k represents the number of elements in the current input corpus.

[0088] Furthermore, when training the event extraction model, the gradient error backpropagation algorithm is used to jointly train the first feature extraction network and the second feature extraction network.

[0089] Step S4: Set a new classification threshold, input the input corpus into the trained event extraction model, and predict the event type based on conditional probability.

[0090] It should be noted that when using the trained event extraction model to extract events, the Softmax layer is used to calculate the prediction probability, and the event type corresponding to the highest probability is the final prediction result of the event extraction model.

[0091] Specifically, the output of the event extraction model is normalized using a Softmax layer:

[0092]

[0093] Where z represents the input vector of the Softmax layer, i.e., the output vector of the event extraction model. i z represents the value corresponding to the i-th position of the input vector. q This represents the value corresponding to the q-th bit of the input vector;

[0094] Modify the classification threshold for each event category (effectively improving the robustness of event extraction):

[0095]

[0096] Where, θ p Let m represent the classification threshold for the current p-th event, where p ∈ m+1; treat non-event categories as a special event category, and since m represents the number of event categories, m+1 represents the total number of event categories.

[0097] Based on the obtained probability distribution, determine whether the predicted probability of the event type with the highest predicted probability is greater than the classification threshold of the current category event. If it is greater, the event extraction model predicts the current category event as the output. Otherwise, delete the highest probability value in the current probability distribution and use the remaining part as the new probability distribution output. Repeat this step until the predicted probability is greater than the classification threshold of the current category event.

[0098] This Chinese domain event extraction method employs a military BERT model, syntactic dependency parsing, and part-of-speech tagging to preprocess the input corpus of the event extraction dataset, obtaining raw feature representations and enhancing the contextual semantic representation capability of the event extraction model. Furthermore, this method constructs an event extraction model comprising a first feature extraction network and a second feature extraction network, and effectively improves the accuracy of event extraction by fusing the features extracted by each feature extraction network.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The embodiments described above are merely specific and detailed examples of the embodiments described in this application, and should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the appended claims.

Claims

1. A method for extracting events in the Chinese domain, characterized in that: The Chinese domain event extraction method includes: Original domain corpora are selected from the domain database to construct a domain dataset and a professional vocabulary, and the militaryBERT model is trained. The input corpus of the event extraction dataset was preprocessed using the militaryBERT model, the dependency parsing tool DDParser, and the part-of-speech tagging tool Jieba to obtain the final raw feature representations. An event extraction model consisting of two different feature extraction networks is constructed. The original feature representation and the input corpus are sequentially input into the first feature extraction network and the second feature extraction network. The corresponding loss weights are adjusted according to the number of each event category. The event extraction model is trained using the gradient error backpropagation algorithm. Set a new classification threshold, input the input corpus into the trained event extraction model, and predict the event type based on conditional probability; The design of the first feature extraction network is as follows: The first layer is a bidirectional LSTM layer, which takes the original feature representation as input and captures the temporal information of the target word context; The second layer is the first fully connected layer, which is used to integrate the extracted deep text features and convert the deep features into a regular matrix; The third layer is a multi-layer graph neural network. It updates the edges of the graph neural network by updating the weight matrix between each character in the dependency parsing tree. The number of graph neural networks used in the current layer is adjusted by the following formula: Where A represents the adjacency matrix; A ij W indicates that there is a one-way edge between node i and node j. l Let b represent the weight matrix of the l-th layer of the graph neural network. l This represents the bias matrix of the l-th layer of the graph neural network. This represents the hidden state of the (l-1)th layer graph neural network with respect to node j. Let σ represent the hidden state of the l-th layer graph neural network with respect to node j, σ represent the ReLU activation function, and pooling represent the pooling operation. The fourth layer is the second fully connected layer, which is used to compress the output dimension of the multi-layer graph neural network to the size of the number of types plus one. The design of the second feature extraction network is as follows: The first layer is the militaryBERT model, which takes the input corpus of the event extraction dataset as input, uses a multi-head self-attention mechanism to capture word-level semantic features, and converts them into high-dimensional dynamic word vectors; The second layer is the CRF layer, which learns the relationships between tags; The outputs of the first feature extraction network and the second feature extraction network are concatenated. Furthermore, when training the event extraction model, the gradient error backpropagation algorithm is used to jointly train the first feature extraction network and the second feature extraction network.

2. The Chinese domain event extraction method as described in claim 1, characterized in that: The input corpus of the event extraction dataset is preprocessed using the military BERT model to obtain the final raw feature representation, including: The militaryBERT model is used to perform entity recognition on the input corpus of the event extraction dataset. Based on the recognition results, markers are added before and after the corresponding entities in the input corpus, and the added markers correspond one-to-one with the recognized entity types. The militaryBERT model is used to encode the current character sequence to obtain character-level character encoding vectors. Using the dependency parsing tool DDParser, a dependency parsing tree between syntactic units is constructed. Each character of the current dependent word is pointed to the string sequence id of the dominant word. The obtained dependency tag information is encoded to obtain a character-level dependency tag encoding vector. The part-of-speech tagging tool Jieba is used to perform part-of-speech tagging on the input corpus. The part-of-speech tag of the current word is assigned to each character that makes up the current word. The obtained part-of-speech tag information is encoded to obtain a character-level part-of-speech tag encoding vector. The character-level character encoding vector, character-level dependency tag encoding vector, and character-level part-of-speech tag encoding vector are concatenated according to the character sequence to obtain the final original feature representation.

3. The Chinese domain event extraction method as described in claim 1, characterized in that: The adjustment of the corresponding loss weight based on the number of each event category includes: The overall loss function of the event extraction model is: loss = αloss1 + (1-α)loss2 Where α represents the hyperparameter, loss1 represents the loss of the first feature extraction network, and loss2 represents the loss of the second feature extraction network; Preset loss weights based on the number of events in each category: Among them, W p This represents the loss weight for the current p-th type of event, where n p This represents the number of events of type p, and m represents the number of event types. And the losses of the first feature extraction network and the second feature extraction network are: Where W represents the loss weight of the event category to which the current input corpus belongs, and X... i This indicates the actual event labels in the current input corpus. This represents the event label predicted from the current input corpus, and k represents the number of elements in the current input corpus.

4. The Chinese domain event extraction method as described in claim 3, characterized in that: The step of setting a new classification threshold, inputting the input corpus into the trained event extraction model, and predicting the event type based on conditional probability includes: The output of the event extraction model is normalized using a Softmax layer: Where z represents the input vector of the Softmax layer, z i z represents the value corresponding to the i-th position of the input vector. q This represents the value corresponding to the q-th bit of the input vector; Modify the classification threshold for each event category: Where, θ p This represents the classification threshold for the current p-th type of event, where p∈m+1; Based on the obtained probability distribution, determine whether the predicted probability of the event type with the highest predicted probability is greater than the classification threshold of the current category event. If it is greater, the event extraction model predicts the current category event as the output. Otherwise, delete the highest probability value in the current probability distribution and use the remaining part as the new probability distribution output. Repeat this step until the predicted probability is greater than the classification threshold of the current category event.

Images