A pre-trained model-based event extraction method for the field of documents
By using a pre-trained model-based event extraction method, which utilizes crawled document data and a BERT+Softmax neural network model, the problem of poor event extraction performance in existing technologies is solved, achieving higher accuracy in event extraction and stronger model adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING INST OF COMP TECH & APPL
- Filing Date
- 2022-05-23
- Publication Date
- 2026-04-24
AI Technical Summary
In existing event extraction methods based on neural network models, entity recognition technology is often used to identify event argument information, resulting in poor performance of event extraction tasks.
An event extraction method based on a pre-trained model is adopted. By crawling document data, constructing a table of trigger words and event arguments, and building a reading comprehension neural network model of Bert+Softmax, trigger word recognition and event argument information extraction are performed. The Adam algorithm is used to optimize the model parameters.
It improves the accuracy of event extraction and the generalization ability of the model, enabling it to exhibit human-like understanding in document event extraction and adapt to event extraction tasks of different types of documents.
Smart Images
Figure CN115203507B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of document intelligence technology application, specifically involving an event extraction method based on a pre-trained model for the document field. Background Technology
[0002] With the increasing application of artificial intelligence and big data technologies, intelligent processing technology for massive documents has become a hot research topic. Faced with a large number of unstructured documents, it is difficult to extract key information and further utilize this information efficiently. Documents contain a large amount of related element information, such as entities, relationships, and events. Key information can be obtained by extracting entity, relationship, and event information.
[0003] Currently, there are two methods for implementing event extraction tasks: template matching based on feature extraction and model learning based on neural networks. Template matching based on feature extraction requires developing templates that satisfy the target event information from a specific domain knowledge base, and then completing the event extraction task through template matching. This method can achieve high accuracy within a specific domain, but its system has poor portability and generalization ability. When the usage scenario changes, it is often necessary to develop new templates based on the new domain scenario knowledge. However, developing new templates is time-consuming and labor-intensive, and also requires guidance from experts in the new domain, which is clearly not desirable. Model learning based on neural networks transforms the event extraction task into the task of identifying event trigger words and corresponding event argument parameters. This method can automatically learn event features, mainly through the construction of the neural network model. Existing event extraction methods based on neural network models typically use entity recognition technology for event argument information recognition. Because this technology relies heavily on entity recognition performance, the performance of the event extraction task is poor. Therefore, to solve this problem, this invention uses a natural language reading comprehension approach to build a neural network model to realize the tasks of event trigger word recognition and event argument information extraction, thus completing the event extraction task. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] The technical problem to be solved by this invention is how to provide an event extraction method based on a pre-trained model for the field of documents, so as to solve the problem that the event argument information recognition of existing event extraction methods based on neural network models usually adopts entity recognition technology, resulting in poor performance of event extraction tasks.
[0006] (II) Technical Solution
[0007] To address the aforementioned technical problems, this invention proposes an event extraction method based on a pre-trained model for the document domain. This method includes the following steps:
[0008] Step S1: Use web crawling technology to crawl publicly available documents and perform noise reduction processing on them, including special symbol processing, dirty data removal, word segmentation and stop word filtering.
[0009] Step S2: Based on the document business requirements, formulate the original trigger vocabulary and event argument list and expand the vocabulary;
[0010] Step S3: Construct question pair templates based on the trigger vocabulary and event argument table, and complete the specified format annotation of the question pair data;
[0011] Step S4: Build a joint neural network model for trigger word recognition and event argument recognition based on the Bert pre-trained language model. Use the Adam algorithm to train and validate the neural network model, optimize the model parameters to make the model converge and save it.
[0012] Step S5: Use the model to predict and deduplicate the trigger words and event argument information respectively, so as to obtain the event extraction results.
[0013] Furthermore, step S1 specifically includes: using web crawler technology to obtain publicly available document text data from document-related websites, using regular expression matching and string processing algorithms to find and delete empty text documents, and processing blank lines and special symbols in the documents, uniformly processing data cleaning, and using the Jieba word segmentation tool and an existing dedicated stop word list to perform word segmentation and stop word filtering on the document data.
[0014] Further, step S2 specifically includes: formulating an original trigger word list and event argument list based on professional knowledge in the document domain, and expanding the word list using a semantic clustering algorithm. The steps for expanding the word list are as follows: S21, loading the word segmentation results of the corpus processed in step S1, and using Word2vec to obtain the word vector representations of the corpus word segmentation; S22, calculating the semantic distance of the word segments based on the K-means clustering algorithm, and obtaining a set of words synonymous with the original trigger words and event arguments; S23, based on the synonym set in step S22, further updating the original trigger word list and event argument list according to human experience, and completing the word list expansion.
[0015] Furthermore, step S3 specifically includes: based on the trigger word list and event argument list expanded in step S2 above, constructing a question pair template for the reading comprehension task. The template is: [CLS] question [SEP] text of the answer to the question [SEP]. The question revolves around wh_word, that is, the question is set through what, who, and where.
[0016] Furthermore, there may be multiple event arguments for the same trigger word, or multiple trigger words for the same event argument, with one trigger word and a set of event arguments corresponding to multiple question pairs.
[0017] Furthermore, the question template data is labeled according to the specified format, and the start and end positions of the corresponding answer in the text and whether there is answer information are obtained by using a regular expression matching algorithm, thereby completing the corpus labeling.
[0018] Further, step S4 specifically includes: constructing a reading comprehension neural network model based on Bert+Softmax, which consists of two sub-tasks: trigger word recognition and event argument information extraction. The labeled corpus obtained in step S3 is divided into training data, validation data, and test data in a ratio of 7:2:1. The training dataset and validation dataset are respectively input into the Bert+Softmax model for training and validation. Since the model is jointly generated by two sub-tasks, the joint loss function of the model is the sum of the loss functions of the two sub-tasks. The joint loss function is optimized by adjusting the network parameters to make the model reach a convergent state. Finally, the model parameters at convergence are saved.
[0019] Furthermore, the neural network model consists of two sub-tasks: trigger word recognition and event argument information extraction. The model first performs word vector representation based on the BERT pre-trained model, then classifies trigger words by adding a Softmax layer, and predicts the start and end positions of event arguments through a binary classification network. Finally, the trigger word multi-classification task and the argument prediction task are merged to complete the joint training of the model. The BERT pre-trained model is composed of 12 Transformer Encoder modules and uses a bidirectional attention mechanism to train the model with Masked LM and Next Sentence Prediction as the target tasks.
[0020] Furthermore, the training and validation sets are input into the model for training and validation. The Adam algorithm is used to adjust and optimize the parameters accordingly. When the model reaches convergence, the corresponding network parameters are saved. The model's loss function is the sum of the loss functions of the two sub-tasks, and the joint loss function is set as: Loss = λ * L trigger +L arguments , where L trigger L is the loss function for triggering word recognition tasks. arguments The loss function for the event argument information extraction task is defined by the parameter λ, which adjusts the rate of descent of the joint loss function of the model. The value of parameter λ depends on the number of trigger words and event argument categories, λ = num. trigger / num arguments .
[0021] Further, step S5 specifically includes: inputting the test set data into the model for trigger word recognition and event argument information extraction prediction, and using a text matching algorithm to deduplicate the trigger words and event argument information, and finally obtaining the event extraction result.
[0022] (III) Beneficial Effects
[0023] This invention proposes a pre-trained model-based event extraction method for the document domain. The innovation lies in using machine reading comprehension tasks within the field of natural language processing for event extraction. It employs a pre-trained language model with a self-attention mechanism for word vector representation, and simultaneously constructs a multi-classification model for trigger word recognition and a binary classification model for predicting the start and end positions of event arguments. These network parameters are jointly adjusted to achieve model convergence. This interactive method of extracting semantic features from text endows the model with human-like text comprehension capabilities, enabling more accurate event extraction. Attached Figure Description
[0024] Figure 1 This is a flowchart of the document event extraction method of the present invention;
[0025] Figure 2 This is a structural diagram of the Bert pre-trained neural network model of this invention. Detailed Implementation
[0026] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.
[0027] This invention aims to address the challenges faced by current event extraction tasks by proposing a pre-trained model-based event extraction method for the document domain.
[0028] The BERT model is a bidirectional self-attention mechanism model that generates word vectors by extracting contextual features. It is a dynamic word vector representation method that can solve the problem of not being able to represent polysemy in traditional static word embedding technologies such as Word2vec and FastText. This allows for a more flexible and richer acquisition of word semantic information. Downstream reading comprehension task models are built by connecting a Softmax layer on top of the BERT model layer. This model is an end-to-end model, mainly composed of two sub-tasks: trigger word recognition and event argument information extraction.
[0029] This invention provides an event extraction method based on a pre-trained model for the document domain, comprising: Step 1, crawling publicly available document corpora using web scraping technology, and cleaning and preprocessing the data; Step 2, constructing an event trigger word and argument information table based on professional knowledge and further expanding it; Step 3, formulating question-pair templates based on the trigger word table and corresponding event argument information, and completing template data annotation according to a specified format (annotating the start and end positions of the search answer information); Step 4, building a BERT+Softmax neural network model that jointly performs trigger word recognition and event argument information extraction tasks. This model uses the BERT baseline model for word vector representation, and the BERT model consists of a Transformer Encoder module and employs a bidirectional self-attention mechanism for semantic feature extraction, which greatly enhances the model's semantic understanding of document context information; Step 5, predicting the event extraction result by loading model parameters onto the prediction dataset. This invention transforms the event extraction task into a machine reading comprehension task, improving the accuracy of event extraction, giving the model human-like comprehension ability, and enhancing the model's generalization ability and flexibility, achieving good results in document event extraction applications.
[0030] The technical solution described in this invention requires the following steps:
[0031] Step S1: Use web crawling technology to crawl publicly available documents and perform noise reduction processing on them, including special symbol processing, dirty data removal, word segmentation and stop word filtering, etc.
[0032] Step S2: Based on the document business requirements, formulate the original trigger vocabulary and event argument list and expand the vocabulary;
[0033] Step S3: Construct question pair templates based on the trigger vocabulary and event argument table, and complete the specified format annotation of the question pair data;
[0034] Step S4: Build a joint neural network model for trigger word recognition and event argument recognition based on the Bert pre-trained language model. Use the Adam algorithm to train and validate the neural network model, optimize the model parameters to make the model converge and save it.
[0035] Step S5: Use the model to predict and deduplicate the trigger words and event argument information respectively, so as to obtain the event extraction results.
[0036] In one embodiment of the document event extraction method of the present invention, step S1 involves using web crawling technology to obtain publicly available document text data from document-related websites, searching and deleting empty text documents based on regular expression matching and string processing algorithms, and processing blank lines, special symbols, and characters in the documents, as well as uniformly processing Chinese characters and other data cleaning. The document data is then segmented and filtered for words using the Jieba word segmentation tool and an existing dedicated stop word list.
[0037] In one embodiment of the document event extraction method of the present invention, in step S2, an original trigger word list and event argument list are formulated based on professional knowledge in the document domain, and the word list is expanded using a semantic clustering algorithm. The word list expansion implementation steps are as follows: S21, load the word segmentation results of the corpus processed in step S1, and obtain the word vector representations of the corpus word segmentation using Word2vec; S22, calculate the semantic distance of the word segments based on the K-means clustering algorithm, and obtain a set of words synonymous with the original trigger words and event arguments; S23, based on the synonym set in step S22, further update the original trigger word list and event argument list according to manual experience, completing the word list expansion.
[0038] In one embodiment of the document event extraction method of the present invention, in step S3, a question pair template for a reading comprehension task is constructed based on the trigger word list and event argument list expanded in step S2 above. The template is: [CLS] Question [SEP] Text of the answer to the question [SEP]. The questions mainly revolve around wh_word, that is, questions are set through what, who, and where.
[0039] For the same trigger word, there can be multiple event arguments, or for the same event argument, there can be multiple trigger words. A trigger word and a set of event arguments can correspond to multiple question pairs. This template expansion strategy can solve the problem of sparsity in the model corpus data of massive documents.
[0040] The question template data is labeled according to the specified format. The main method is to use regular expression matching algorithm to obtain information such as the start and end positions of the corresponding answer in the text and whether there is an answer, so as to complete the corpus labeling.
[0041] In one embodiment of the document event extraction method of the present invention, in step S4, a reading comprehension neural network model based on Bert+Softmax is constructed. This model consists of two sub-tasks: trigger word recognition and event argument information extraction. The labeled corpus obtained in step S3 is divided into training data, validation data, and test data in a ratio of 7:2:1. The training dataset and the validation dataset are respectively input into the Bert+Softmax model for training and validation. Since the model is jointly generated by the two sub-tasks, the joint loss function of the model is the sum of the loss functions of the two sub-tasks. The joint loss function is optimized by adjusting the network parameters to make the model reach a convergent state. Finally, the model parameters at convergence are saved.
[0042] In one embodiment of the document event extraction method of the present invention, in step S5, the test set data is input into the model for trigger word recognition and event argument information extraction prediction, and the trigger words and event argument information are deduplicated using a text matching algorithm to finally obtain the event extraction result.
[0043] Figure 1 Here is a flowchart of a document-based event extraction method, see reference. Figure 1 The embodiments of the technical solution of the present invention will be further described below:
[0044] Example 1: Document Acquisition and Preprocessing (Noise Reduction)
[0045] This embodiment includes the following steps:
[0046] A. Web crawling technology was used to crawl the original text corpus of publicly available documents, mainly including webpage URL crawling, webpage data parsing, and document corpus storage. First, the Scrapy crawling framework was used to crawl the structured data of publicly available document websites; second, the BeautifulSoup4 library was used to parse the webpage data; finally, the document corpus was saved in JSON format.
[0047] B. Preprocessing of the original document corpus: First, a text matching algorithm is used to find and delete empty data and corpus data that does not contain the target information; second, special symbols, inconsistent punctuation, extra blank lines and spaces, and potentially unparsed web page data and other similar noise data in the document are processed; finally, the Jieba word segmentation tool is used to segment the corpus and filter stop words (a dedicated word segmentation dictionary and stop word dictionary need to be loaded).
[0048] Example 2: An initial trigger word list and event argument list are developed by combining professional knowledge of written documents, and the word list is expanded using the available written document corpus obtained in Example 1. Event arguments corresponding to different trigger words are developed based on the available written document corpus. The semantic distance is calculated using the K-means clustering algorithm to obtain the synonym set corresponding to the original word list. The trigger word list and event argument list are expanded using empirical and professional knowledge. This word list expansion strategy mainly solves the problem of data sparsity in the model, which is beneficial to enhancing the model's learning ability.
[0049] Example 3: Question pair template construction and model training corpus annotation based on trigger word list and event argument list.
[0050] This embodiment includes the following steps:
[0051] A. Template construction for document corpus questions. Furthermore, similar question templates are developed for synonyms of trigger words. Based on the templates constructed from the model input, they are divided using [SEP], with the first half representing the question and the second half representing the text of the answer to the question. This input aims to teach the model the ability to find the corresponding answer location in the text, allowing the model to identify trigger words in the text.
[0052] B. Use a text matching algorithm to obtain information such as the start and end positions of answer items in the document and whether an answer exists in the template data, and complete the question labeling of the template according to the model training data format. If the event extraction of multiple sample data simultaneously results in a single sample being too long, the long sample data can be divided into multiple subsamples. Events from the subsamples can be extracted and merged to obtain all events of the long sample data. However, if event extraction is performed on multiple samples one by one, ID labeling is not necessary.
[0053] Example 4: An end-to-end reading comprehension neural network model is built based on the BERT baseline model. This model consists of two sub-tasks: trigger word recognition and event argument information extraction. For example... Figure 2As shown, the model first uses a BERT pre-trained model for word vector representation, then adds a Softmax layer for trigger word classification, and uses a binary classification network for event argument start and end position prediction. Finally, the trigger word multi-classification task and argument prediction task are merged to complete the joint training of the model. The BERT pre-trained model consists of 12 Transformer Encoder modules and uses a bidirectional attention mechanism with Masked Learning Model (LM) and Next Sentence Prediction as target tasks for model training. This pre-trained language model can more deeply mine and understand the semantic information of documents and dynamically represent word vectors, solving the semantic representation of the same word in different scenarios. Meanwhile, since each document contains multiple trigger words, document trigger word recognition is a multi-classification task, requiring the addition of a Softmax layer to the BERT model to complete trigger word classification and prediction.
[0054] The training and validation datasets are input into the model for training and validation. The Adam algorithm is used to adjust and optimize the parameters. When the model reaches convergence, the corresponding network parameters are saved. Since the model is generated jointly by two sub-tasks, the model's loss function is the sum of the loss functions of the two sub-tasks. The joint loss function is set as: Loss = λ * L trigger +L arguments , where L trigger L is the loss function for triggering word recognition tasks. arguments For the event argument information extraction task, since the convergence rates of the trigger word recognition and event argument information subtasks are inconsistent, a parameter λ needs to be added to adjust the descent rate of the joint loss function of the model. The value of parameter λ depends on the number of trigger words and event argument categories (λ = num). trigger / num arguments ).
[0055] Example 5:
[0056] The test set data is used as input to the model for trigger word recognition and event argument extraction prediction. The trigger words and event arguments are deduplicated using a text matching algorithm to obtain the final event extraction results. The deduplication strategy is as follows: the trigger words and corresponding arguments predicted for each sample are stored in their respective lists. Each time a trigger word or corresponding argument is stored, it is first checked whether the corresponding list already contains the event trigger word or corresponding argument to be stored. If it does not exist, it is stored in the list; otherwise, it is not stored.
[0057] Example 6:
[0058] An event extraction method based on a pre-trained model for the document domain includes the following steps:
[0059] Step 1, document acquisition and preprocessing, mainly involves crawling publicly available document corpora and denoising the corpora, including deleting blank lines or special symbols, handling inconsistencies between Chinese and English characters, and processing some unparsed web page data. Finally, the corpora are processed by word segmentation and stop word filtering.
[0060] Step 2: Based on the document business requirements, formulate the original trigger word list and event argument list and expand the word list. The word list expansion strategy uses the K-means clustering algorithm to calculate the semantic distance of words to obtain the original word list synonym set and update the word list.
[0061] Step 3: Construct question pair templates based on the trigger word list and event argument list. The main focus is on the 4Ws (what, who, which, where) to formulate questions corresponding to the same trigger word and event argument, thereby generating question pair templates. At the same time, the question pair data is labeled, including information such as the start and end positions of the corresponding answers in the text and whether the answers exist.
[0062] Step 4: Build a neural network model that combines trigger word recognition and event argument extraction tasks based on the BERT pre-trained language model. Build a Softmax layer on top of the BERT model layer to optimize the joint loss function for trigger word recognition and argument extraction, and save the hyperparameters when the model converges.
[0063] Step 5: Load the saved model parameters, use the test set data to predict the model output, and perform deduplication on the predicted event trigger words and event arguments to finally obtain the event extraction results.
[0064] Furthermore, web crawling and web page data parsing technologies were used to obtain publicly available document corpus data. Regular expression matching and string processing algorithms were used to find and delete empty text documents. Data cleaning was performed to address issues such as blank lines, special symbols, and inconsistencies between Chinese and English characters in the documents. In addition, domain-specific expertise was used to construct a dedicated word segmentation dictionary and stop word list. The Jieba word segmentation tool was used to segment and filter the corpus data.
[0065] Furthermore, based on domain-specific knowledge, an initial trigger word list and event argument list were developed, and the word list was expanded using a semantic clustering algorithm. The word list expansion process involved the following steps: A. Loading the word segmentation results from step one and using Word2vec to obtain the word vector representations from the segmented corpus; B. Calculating semantic distance using the K-means clustering algorithm to obtain a set of words synonymous with the initial trigger words and event arguments; C. Based on the synonym set from step B, further updating the initial trigger word list and event argument list based on manual experience to complete the word list expansion.
[0066] Furthermore, based on the updated trigger word list and event argument list from step two above, a question pair template for the reading comprehension task is constructed. The construction of the question pair template mainly involves formulating questions corresponding to trigger words and event arguments. Question formulation revolves around the wh_word, i.e., setting questions using what, who, and where.
[0067] For the same trigger word, there can be multiple event arguments, or for the same event argument, there can be multiple trigger words. A trigger word and a set of event arguments can correspond to multiple question pairs. This question pair template expansion strategy can solve the problem of sparsity in the model corpus data of massive documents.
[0068] The question template data is labeled according to the specified format. The main method is to use regular expression matching algorithm to obtain information such as the start and end positions of the corresponding answer in the text and whether there is an answer, so as to complete the corpus labeling.
[0069] Furthermore, a reading comprehension neural network model based on BERT+Softmax is constructed. This model consists of two sub-tasks: trigger word recognition and event argument information extraction. The labeled corpus obtained in step three is divided into training, validation, and test data in a ratio of 7:2:1. The training and validation datasets are respectively input into the BERT+Softmax model for training and validation. Since the model is jointly generated by the two sub-tasks, the joint loss function of the model is the sum of the loss functions of the two sub-tasks. The joint loss function is optimized by adjusting the network parameters to make the model converge. Finally, the model parameters at convergence are saved.
[0070] Furthermore, the test set data is input into the model for trigger word recognition and event argument information extraction prediction. A text matching algorithm is then used to deduplicate the trigger words and event argument information to finally obtain the event extraction results.
[0071] This invention addresses the shortcomings of existing event extraction methods, achieving the following improvements: Traditional pattern-matching-based event extraction techniques heavily rely on rule templates, resulting in poor scalability and portability, making them unsuitable for extracting events from documents with diverse and complex case scenarios. Furthermore, while existing deep learning-based event extraction techniques can overcome the limitations of traditional pattern-matching, they require specialized knowledge to develop and expand the trigger words and event argument vocabularies needed for event extraction, and they cannot accurately identify document corpora not trained on by the model, leading to weak generalization ability and an inability to better adapt to different document types. This invention, starting from a deep understanding of document semantics, uses a BERT pre-trained language model to obtain word vector representations of documents and adds a Softmax layer to the BERT model to construct a reading comprehension-like neural network model. This gives the model human-like comprehension capabilities, enabling deeper word extraction of document features and more accurate event information extraction. This pre-trained model-based event extraction method for the document domain is better applicable to different document types and demonstrates excellent performance in event extraction tasks.
[0072] The innovation of this invention lies in using machine reading comprehension tasks within the field of natural language processing for event extraction. It employs a pre-trained language model with a self-attention mechanism for word vector representation, and simultaneously constructs a multi-classification model for trigger word recognition and a binary classification model for predicting the start and end positions of event argument information. These network parameters are jointly adjusted to achieve model convergence. This interactive method of extracting semantic features from text endows the model with human-like text comprehension capabilities, enabling more accurate event extraction.
[0073] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for event extraction based on a pre-trained model for the document domain, characterized in that, The method includes the following steps: Step S1: Use web crawling technology to crawl publicly available documents and perform noise reduction processing on them, including special symbol processing, dirty data removal, word segmentation and stop word filtering. Step S2: Based on the document business requirements, formulate the original trigger vocabulary and event argument list and expand the vocabulary; Step S3: Construct question pair templates based on the trigger vocabulary and event argument table, and complete the specified format annotation of the question pair data; Step S4: Build a joint neural network model for trigger word recognition and event argument recognition based on the Bert pre-trained language model. Use the Adam algorithm to train and validate the neural network model, optimize the model parameters to make the model converge and save it. Step S5: Use the model to predict and deduplicate the trigger words and event argument information respectively, so as to obtain the event extraction results; in, The neural network model consists of two sub-tasks: trigger word recognition and event argument information extraction. The model first represents word vectors based on a BERT pre-trained model, then classifies trigger words by adding a Softmax layer, and predicts the start and end positions of event arguments using a binary classification network. Finally, the trigger word multi-classification task and the argument prediction task are merged to complete the joint training of the model. The BERT pre-trained model is composed of 12 Transformer Encoder modules and uses a bidirectional attention mechanism with Masked LM and Next Sentence Prediction as the target tasks for model training. The training and validation sets are input into the model for training and validation. The Adam algorithm is used to adjust and optimize the parameters. When the model reaches convergence, the corresponding network parameters are saved. The model's loss function is the sum of the loss functions of the two sub-tasks, and the joint loss function is set as follows: ,in, For the loss function of the trigger word recognition task, For the task of extracting event argument information, the loss function is... The parameters are used to adjust the rate of descent of the model's joint loss function, while the parameters... The value depends on the number of trigger words and event argument categories. .
2. The event extraction method based on a pre-trained model for the document domain as described in claim 1, characterized in that, Step S1 specifically includes: using web crawler technology to obtain publicly available document text data from document-related websites; using regular expression matching and string processing algorithms to find and delete empty text documents; processing blank lines and special symbols in the documents; uniformly processing data cleaning; and using the Jieba word segmentation tool and an existing dedicated stop word list to perform word segmentation and stop word filtering on the document data.
3. The event extraction method based on a pre-trained model for the document domain as described in claim 2, characterized in that, Step S2 specifically includes: developing an original trigger word list and event argument list based on professional knowledge in the document domain, and expanding the word list using a semantic clustering algorithm. The word list expansion steps are as follows: S21, loading the word segmentation results of the corpus processed in step S1, and using Word2vec to obtain the word vector representations of the corpus word segmentation; S22, calculating the semantic distance of the word segments based on the K-means clustering algorithm, and obtaining a set of words synonymous with the original trigger words and event arguments; S23, based on the synonym set in step S22, further updating the original trigger word list and event argument list based on manual experience to complete the word list expansion.
4. The event extraction method based on a pre-trained model for the document domain as described in claim 3, characterized in that, Step S3 specifically includes: based on the trigger word list and event argument list expanded in step S2 above, constructing a question pair template for the reading comprehension task. The template is: [CLS] Question [SEP] Text of the answer to the question [SEP]. The questions revolve around wh_word, that is, the questions are set through what, who, and where.
5. The event extraction method based on a pre-trained model for the document domain as described in claim 4, characterized in that, For the same trigger word, there may be multiple event arguments, or for the same event argument, there may be multiple trigger words. One trigger word and a set of event arguments may correspond to multiple question pairs.
6. The event extraction method based on a pre-trained model for the document domain as described in claim 4, characterized in that, The question template data is labeled according to the specified format, and the start and end positions of the corresponding answer in the text and whether there is answer information are obtained by using a regular expression matching algorithm, thereby completing the corpus labeling.
7. The event extraction method based on a pre-trained model for the document domain as described in any one of claims 1-6, characterized in that, Step S4 specifically includes: constructing a reading comprehension neural network model based on Bert+Softmax. This model consists of two sub-tasks: trigger word recognition and event argument information extraction. The labeled corpus obtained in step S3 is divided into training data, validation data, and test data in a ratio of 7:2:
1. The training dataset and validation dataset are respectively input into the Bert+Softmax model for training and validation. Since this model is jointly generated by two sub-tasks, the joint loss function of the model is the sum of the loss functions of the two sub-tasks. The joint loss function is optimized by adjusting the network parameters to make the model reach a convergent state. Finally, the model parameters at convergence are saved.
8. The event extraction method based on a pre-trained model for the document domain as described in claim 7, characterized in that, Step S5 specifically includes: inputting test set data into the model for trigger word recognition and event argument information extraction and prediction, and using a text matching algorithm to deduplicate the trigger words and event argument information, and finally obtaining the event extraction result.