Cross-border ethnic culture event extraction method and device based on event pre-training
By constructing event structure diagrams in the field of cross -border ethnic culture, and combining comparative learning pre -training models, the incidents that are difficult to classify caused by complexity between events and factoring have significantly improved the accuracy and effect of event extraction.
Patent Information
- Application Number
- CN202410803883.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2025-01-28
AI Technical Summary
The event extraction tasks in the field of cross-border ethnic culture face the complex relationships such as juxtaposition and subordinate relationships between event elements, which lead to the difficulty of classification of events.
The method of cross -border ethnic cultural events based on event pre -training is adopted, and the structural information of cross -border national cultural events is used to integrate the structural information of the event, and use a comparative learning pre -training text encoder and event structure diagram to enhance the characteristics of the event.
Effectively alleviated the problem of poor cross -border national cultural events, and improved the classification accuracy and drawing effect of the event elements.
Smart Images
Figure CN119377398A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method and device for extracting cross-border ethnic cultural events based on event pre-training, and belongs to the technical field of natural language processing. Background Art
[0002] Event extraction is an important research direction in the field of natural language processing (NLP), which aims to extract words that can express event information from unstructured or semi-structured texts, such as the time and place of the event. At present, event extraction methods in general fields have achieved good results, while event extraction tasks in the field of cross-border ethnic culture still face many problems to be solved. First, the text in the field of cross-border ethnic culture contains many words with certain domain characteristics, and these words are mostly entity words specific to some fields. Among them, some domain entity words are event elements that need to be extracted in the cross-border ethnic cultural event extraction task. Better representation of these domain words can effectively improve the effects of cross-border ethnic cultural event extraction. Second, events in the field of cross-border ethnic culture are mostly events related to ethnic culture. An event is often composed of multiple different parts, and there are parallel, modified, subordinate and other relationships between these parts, which poses a great challenge to the cross-border ethnic cultural event extraction task. In view of the problems existing in the event extraction task in the field of cross-border ethnic culture, the present invention proposes a cross-border ethnic cultural event extraction method based on event pre-training. Summary of the invention
[0003] The present invention provides a method and device for extracting cross-border ethnic cultural events based on event pre-training, so as to solve the problem that the events are difficult to classify because there are parallel and subordinate relationships among event elements in the cross-border ethnic cultural field.
[0004] The technical solution of the present invention is: in the first aspect, the present invention provides a method for extracting cross-border ethnic cultural events based on event pre-training, and the specific steps of the method are as follows:
[0005] Step 1. Construction of a cross-border ethnic cultural event extraction dataset: Due to the lack of a public cross-border ethnic document event extraction dataset, the present invention collects network resources to construct relevant data of cross-border ethnic groups such as Dai, Yi, Miao, and Zhuang, and processes the data to eventually construct a cross-border ethnic cultural event extraction dataset of 15,000 items.
[0006] Step 2, constructing a structural diagram of cross-border ethnic cultural events: In view of the fact that event elements in the field of cross-border ethnic culture are difficult to classify due to the existence of parallel and subordinate relationships among them, the present invention uses semantic dependency analysis to construct a semantic structural diagram of events, and alleviates the problem of difficulty in classifying event elements by displaying and integrating event structure information.
[0007] Step 3. Construction of a cross-border ethnic cultural event extraction model based on event pre-training: The present invention further enhances the cross-border ethnic cultural event representation by explicitly integrating event structure information into the cross-border ethnic cultural event representation, and pre-trains a text encoder and an event structure graph encoder based on contrastive learning, thereby alleviating the problem of poor cross-border ethnic cultural event extraction results.
[0008] Step 4. Extraction of cross-border ethnic cultural events: Save the trained event extraction model, encapsulate it and deploy it on the server, establish an API to allow other applications to interact with the model, and realize the online extraction of cross-border ethnic cultural events.
[0009] As a preferred embodiment of the present invention, the specific steps of Step 1 are:
[0010] Step 1.1. In order to solve the problem of the lack of public cross-border ethnic document event extraction data sets on the Internet, the present invention obtains relevant data of cross-border ethnic groups such as Dai, Yi, Miao, Zhuang, etc. through media such as Wikipedia, Twitter, and Kunong Lisu Nationality Network, and a total of more than 20,000 relevant documents are collected.
[0011] Step 1.2, cross-border ethnic cultural text data are mostly events related to dance art, festival activities, historical legends, etc., in order to match the document content with the event type. By analyzing the obtained cross-border ethnic cultural text data, the present invention defines cross-border ethnic cultural events as six types of events: cultural and artistic events, religious sacrificial events, festival celebration events, historical legend events, food production events, and NONE events. According to the content described in the text, it is classified into one of the six events, where the classification basis and characteristics of each event are shown in Table 1.
[0012] Table 1 Event types and their characteristic descriptions
[0013]
[0014] Step 1.3. Manually use the event annotation tool publicly available at Stanford University to annotate the cross-border ethnic cultural event extraction dataset. First, import the cross-border ethnic cultural text data into the annotation tool to obtain the entity recognition data and part-of-speech analysis of this sentence. According to these part-of-speech analysis and named entity recognition results, annotate the sentence with event information. Annotate the verbs or nouns in the text that may become trigger words as trigger words; annotate the entities related to the events corresponding to the trigger words as event arguments, and annotate the roles of the corresponding event arguments with reference to the entity type. For event information that was not recognized or incorrectly recognized by the annotation tool, these event information were annotated through manual review. Finally, 15,000 high-quality cross-border ethnic cultural event extraction datasets were annotated by manual annotation. For example, the text "Due to work and business, the Zhuang people moved to Guiyang from Southeast Guizhou, South Guizhou and Guangxi, making them the tenth largest minority in Guiyang" can be divided into "historical legend events" through its text content. The annotation tool can identify "migration" as a trigger word, and "Zhuang, Southeast Guizhou, South Guizhou, Guangxi, Guiyang" as event arguments. For the event arguments involved in "migration", the entity type "NATIONALITY" corresponding to "Zhuang" can be used as an event role, and other event arguments such as "Southeast Guizhou, South Guizhou, Guangxi, Guiyang" can be assigned roles according to the characteristics of the event.
[0015] As a preferred embodiment of the present invention, the specific steps of Step 2 are:
[0016] Step 2.1, use the semantic dependency analysis tool to automatically construct the semantic dependency analysis graph of the cross-border ethnic cultural event text. Since the event arguments in the cross-border ethnic cultural event text have relatively complex relationships, such as sequential and parallel relationships, after analyzing their characteristics, a part of the semantic dependency relationships shown in the following table are selected as the edges of the cross-border ethnic cultural event structure graph. The specific types are shown in Table 1.
[0017] Table 2 Selected relationship table
[0018]
[0019] Step 2.2: Select the edges and nodes connected between the edges that match the cross-border ethnic cultural event relationship from the obtained semantic dependency analysis graph to construct a cross-border ethnic cultural event structure graph. The process of obtaining the cross-border ethnic cultural event structure graph can be abstracted as follows: a cross-border ethnic cultural text, after semantic dependency analysis, obtains its event structure graph G cb The specific formula is as follows:
[0020] G cb ={u,v,e|(u,v)∈V cb×V cb ,e∈E sel}
[0021] Among them, u, v represent vocabulary nodes, e represents the edge between nodes, V cb Represents a word node set, E sel Represents the selected set of relations.
[0022] As a preferred embodiment of the present invention, the specific steps of Step 3 are:
[0023] Step 3.1. In order to learn the semantic information of cross-border ethnic cultural events, a pre-trained language model is selected as the text encoder of cross-border ethnic cultural events, and the text encoder is pre-trained based on the contrastive learning method to enhance the semantic representation of cross-border ethnic cultural events.
[0024] Step 3.2. In order to learn the event structure information of cross-border ethnic cultural events, a graph isomorphism neural network is selected as the cross-border ethnic cultural event structure graph encoder, and the event structure graph encoder is pre-trained based on the contrastive learning method to enhance the structural representation of cross-border ethnic cultural events.
[0025] Step 3.3. In order to solve the problem of difficult classification of event elements, the text encoder and the event structure graph encoder are used together as encoders for cross-border ethnic cultural event extraction. The event text encoding representation and the event structure encoding representation are integrated to extract cross-border ethnic cultural events.
[0026] As a preferred embodiment of the present invention, the specific steps of Step 3.1 are:
[0027] Step 3.1.1, use the cross-border ethnic culture dictionary constructed in the previous work as an aid to better segment the cross-border ethnic culture text. Given a cross-border ethnic culture text containing n words s = {w1, w2, ..., w n}, and combine with the domain dictionary to get {w1,w2,...,w n ,v1…vm}, where w represents a high-frequency word and v represents a domain vocabulary. The high-frequency word w is sent to the pre-trained language model (RoBERTa) and the hidden layer vector x of the last layer is selected. w As a representation of cross-border ethnic cultural texts, low-frequency words are randomly initialized for word embedding. The specific formula is as follows:
[0028] x w =RoBERTa(w i )
[0029] x d =random_word2vec(v i )
[0030] Step 3.1.2, in order to make the words in the same event have a closer representation, the distance between the words of different events in the semantic space is widened. For the semantic pre-training of cross-border ethnic cultural events, the present invention constructs a "trigger word-event argument" pair as positive sampling, and randomly replaces the trigger word with other words that have no direct connection with the event argument or randomly replaces the event argument with other words that have no direct connection with the trigger word as negative sampling. Given a relation set E sel and the positive sample set P of a single cross-border ethnic cultural text s The specific formula is as follows:
[0031] E sel ={AGT,DATV,eCOO,eSUCC,EXP,FEAT,ROOT,SCO,TIME}
[0032] P s ={(u,v)|(u,v,r)∈G cb ,r∈E sel}
[0033] Step 3.1.3 In order to learn to distinguish between positive sample word pairs and negative sample word pairs, so that the model can learn better semantic representation of cross-border ethnic cultural events, the training target of the model is defined as the cross entropy loss that can correctly classify positive sample word pairs. The specific formula is as follows:
[0034]
[0035] Among them, (t, a) represents a positive example word pair, x t ,x a Represents the vector representation of trigger word t and event argument a respectively, m t and m a They represent the number of negative samples obtained by randomly replacing trigger words and the number of negative samples obtained by randomly replacing event arguments, respectively. and Respectively represent the vector representation of randomly replacing trigger words or event arguments, and W represents the weight matrix. Then, the loss obtained for each sentence is added together to obtain the overall training goal of event semantic pre-training. The specific formula is as follows:
[0036]
[0037] Among them, θ represents the parameters that need to be trained, S batch Represents a batch of cross-border national cultural texts.
[0038] As a preferred embodiment of the present invention, the specific steps of Step 3.2 are:
[0039] Step 3.2.1: For cross-border ethnic cultural events with complex event element relationships, using graph structures to model them can clearly represent the relationships between event elements. First, given a cross-border ethnic cultural event semantic structure graph g, its encoding can be expressed as the following formula:
[0040] x g =GNN(g,{x w})
[0041] Among them, x w is the node representation initialized by the text encoder. This node initialization method can implicitly align the semantic space of the event semantic representation and the structure representation in the model of this chapter, so that both can better represent cross-border ethnic cultural texts.
[0042] Step3.2.2, given a batch of m event structure graphs {g1,g2,...g m}, where each event structure graph corresponds to a sentence in the unsupervised corpus. For the i-th subgraph g i , randomly select two subgraphs from its subgraph and As positive examples, we randomly sampled subgraphs from other event structure graphs in the batch as negative examples. The subgraph pair consisting of two subgraphs G1-S1 and G1-S2 from the same mother graph G1 was used as a positive example; the subgraph pair consisting of other subgraphs G2-S1 and G2-S2 from the same batch and subgraph G1-S1 was used as a negative example, as shown in the attached figure. Figure 1 Similar to the contrast pre-training of event semantics, the present invention uses a graph encoder to train the subgraph g sub Encoding, then the graph compares the pre-trained loss function L gra It is defined as follows:
[0043]
[0044] Among them, 1 [j≠2i-1] ∈{0,1} is a discriminant function, which takes the value of 1 when j≠2i-1; θ is the parameter that needs to be trained.
[0045] As a preferred embodiment of the present invention, the specific steps of Step 3.3 are:
[0046] Step 3.3.1. Input the text into the pre-trained text encoder to obtain the semantic encoding representation of the text {x w}, and use the Dynamic Multi-pooling (DM) method to integrate it into a unified representation. The specific formula is as follows:
[0047] X s =DM({x w})
[0048] Step 3.3.2: Select the candidate trigger word or event argument node from the event structure graph, and input the subgraph consisting of the candidate trigger word or event argument node and the node with a direct edge between them into the event structure graph encoder to obtain the structural encoding representation X of the text. g The specific formula is as follows:
[0049] X g =Encoder g (g t′,a′ )
[0050] Step 3.3.3, concatenate the semantic coding representation and structural coding representation of the text as the coding representation of the cross-border ethnic cultural event text, and then send it into a multi-layer fully connected neural network, use softmax for normalization, and get the final classification result.
[0051] As a preferred embodiment of the present invention, the specific steps of Step 4 are:
[0052] Step 4.1. Save the trained model as a ".pth" file and deploy it to the local server. Use Sanic technology to convert the model into an application interface.
[0053] Step 4.2: Call the application interface of the model deployed to the local server on the Web side, directly call the model through the web page, and output the retrieved results to the front-end interface for display.
[0054] In a second aspect, the present invention provides a device for extracting cross-border ethnic cultural events based on event pre-training, including a module for executing the method provided in the first aspect.
[0055] The beneficial effects of the present invention are:
[0056] 1. The present invention uses the cross-border ethnic cultural event structure diagram to construct the "trigger word-corresponding event element" word pair as a positive sample, and constructs the negative sample word pair by randomly replacing the "trigger word" or "trigger word corresponding event element" with other words, and uses contrast loss to train the cross-border ethnic cultural event text encoder. Thus, the model learns a better semantic representation of cross-border ethnic cultural events.
[0057] 2. The present invention selects a graph isomorphism neural network as a cross-border ethnic cultural event structure graph encoder, and uses the constructed cross-border ethnic cultural event structure graph to split it into multiple subgraphs. By constructing subgraph pairs as samples and based on the comparative learning method, the graph encoder can better learn the cross-border ethnic cultural event structure relationship. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] Figure 1 This is a diagram of the event extraction model architecture in the present invention;
[0059] Figure 2 It is a flow chart. DETAILED DESCRIPTION
[0060] Example 1: Figure 1-2 As shown, the method for extracting cross-border ethnic cultural events based on event pre-training, the specific steps of the method for extracting cross-border ethnic cultural events based on event pre-training are as follows:
[0061] Step 1. Construction of a cross-border ethnic cultural event extraction dataset: Due to the lack of a public cross-border ethnic document event extraction dataset, the present invention collects network resources to construct relevant data of cross-border ethnic groups such as Dai, Yi, Miao, and Zhuang, and processes the data to eventually construct a cross-border ethnic cultural event extraction dataset of 15,000 items.
[0062] Step 2, constructing a structural diagram of cross-border ethnic cultural events: In view of the fact that event elements in the field of cross-border ethnic culture are difficult to classify due to the existence of parallel and subordinate relationships among them, the present invention uses semantic dependency analysis to construct a semantic structural diagram of events, and alleviates the problem of difficulty in classifying event elements by displaying and integrating event structure information.
[0063] Step 3. Construction of a cross-border ethnic cultural event extraction model based on event pre-training: The present invention further enhances the cross-border ethnic cultural event representation by explicitly integrating event structure information into the cross-border ethnic cultural event representation, and pre-trains a text encoder and an event structure graph encoder based on contrastive learning, thereby alleviating the problem of poor cross-border ethnic cultural event extraction results.
[0064] Step 4. Extraction of cross-border ethnic cultural events: Save the trained event extraction model, encapsulate it and deploy it on the server, establish an API to allow other applications to interact with the model, and realize the online extraction of cross-border ethnic cultural events.
[0065] As a preferred embodiment of the present invention, the specific steps of Step 1 are:
[0066] Step 1.1. In order to solve the problem of the lack of public cross-border ethnic document event extraction data sets on the Internet, the present invention obtains relevant data of cross-border ethnic groups such as Dai, Yi, Miao, Zhuang, etc. through media such as Wikipedia, Twitter, and Kunong Lisu Nationality Network, and a total of more than 20,000 relevant documents are collected.
[0067] Step 1.2, cross-border ethnic cultural text data are mostly events related to dance art, festival activities, historical legends, etc., in order to match the document content with the event type. By analyzing the obtained cross-border ethnic cultural text data, the present invention defines cross-border ethnic cultural events as six types of events: cultural and artistic events, religious sacrificial events, festival celebration events, historical legend events, food production events, and NONE events. According to the content described in the text, it is classified into one of the six events, where the classification basis and characteristics of each event are shown in Table 1.
[0068] Table 1 Event type table
[0069]
[0070] Step 1.3. Manually use the event annotation tool publicly available at Stanford University to annotate the cross-border ethnic cultural event extraction dataset. First, import the cross-border ethnic cultural text data into the annotation tool to obtain the entity recognition data and part-of-speech analysis of this sentence. According to these part-of-speech analysis and named entity recognition results, annotate the sentence with event information. Annotate the verbs or nouns in the text that may become trigger words as trigger words; annotate the entities related to the events corresponding to the trigger words as event arguments, and annotate the roles of the corresponding event arguments with reference to the entity type. For event information that was not recognized or incorrectly recognized by the annotation tool, these event information were annotated through manual review. Finally, 15,000 high-quality cross-border ethnic cultural event extraction datasets were annotated by manual annotation. For example, the text "Due to work and business, the Zhuang people moved to Guiyang from Southeast Guizhou, Qiannan and Guangxi, making them the tenth largest minority in Guiyang" can be divided into "historical legend events" through its text content. The annotation tool can identify "migration" as a trigger word, and "Zhuang, Southeast Guizhou, Qiannan, Guangxi, Guiyang" as event arguments. For the event arguments involved in "migration", the entity type "NATION" corresponding to "Zhuang" can be used as an event role, and other event arguments such as "Southeast Guizhou, Qiannan, Guangxi, Guiyang" can be assigned roles according to the characteristics of the event.
[0071] As a preferred embodiment of the present invention, the specific steps of Step 2 are:
[0072] Step 2.1, use the semantic dependency analysis tool to automatically construct the semantic dependency analysis graph of the cross-border ethnic cultural event text. Since the event arguments in the cross-border ethnic cultural event text have relatively complex relationships, such as sequential and parallel relationships, after analyzing their characteristics, a part of the semantic dependency relationships shown in the following table are selected as the edges of the cross-border ethnic cultural event structure graph. The specific types are shown in Table 1.
[0073] Table 2 Selected relationship table
[0074]
[0075] Step 2.2: Select the edges and nodes connected between the edges that match the cross-border ethnic cultural event relationship from the obtained semantic dependency analysis graph to construct a cross-border ethnic cultural event structure graph. The process of obtaining the cross-border ethnic cultural event structure graph can be abstracted as follows: a cross-border ethnic cultural text, after semantic dependency analysis, obtains its event structure graph G cb The specific formula is as follows:
[0076] G cb ={u,v,e|(u,v)∈V cb ×V cb ,e∈E sel}
[0077] Among them, u, v represent vocabulary nodes, e represents the edge between nodes, V cb Represents a word node set, E sel Represents the selected set of relations.
[0078] As a preferred embodiment of the present invention, the specific steps of Step 3 are:
[0079] Step 3.1. In order to learn the semantic information of cross-border ethnic cultural events, a pre-trained language model is selected as the text encoder of cross-border ethnic cultural events, and the text encoder is pre-trained based on the contrastive learning method to enhance the semantic representation of cross-border ethnic cultural events.
[0080] Step 3.2. In order to learn the event structure information of cross-border ethnic cultural events, a graph isomorphism neural network is selected as the cross-border ethnic cultural event structure graph encoder, and the event structure graph encoder is pre-trained based on the contrastive learning method to enhance the structural representation of cross-border ethnic cultural events.
[0081] Step 3.3. In order to solve the problem of difficult classification of event elements, the text encoder and the event structure graph encoder are used together as encoders for cross-border ethnic cultural event extraction. The event text encoding representation and the event structure encoding representation are integrated to extract cross-border ethnic cultural events.
[0082] As a preferred embodiment of the present invention, the specific steps of Step 3.1 are:
[0083] Step 3.1.1, use the cross-border ethnic culture dictionary constructed in the previous work as an aid to better segment the cross-border ethnic culture text. Given a cross-border ethnic culture text containing n words s = {w1, w2, ..., w n}, and combine with the domain dictionary to get {w1,w2,...,w n ,v1…vm}, where w represents a high-frequency word and v represents a domain vocabulary. The high-frequency word w is sent to the pre-trained language model (RoBERTa) and the hidden layer vector x of the last layer is selected. w As a representation of cross-border ethnic cultural texts, low-frequency words are randomly initialized for word embedding. The specific formula is as follows:
[0084] x w =RoBERTa(w i )
[0085] x d =random_word2vec(v i )
[0086] Step 3.1.2, in order to make the words in the same event have a closer representation, the distance between the words of different events in the semantic space is widened. For the semantic pre-training of cross-border ethnic cultural events, the present invention constructs a "trigger word-event argument" pair as positive sampling, and randomly replaces the trigger word with other words that have no direct connection with the event argument or randomly replaces the event argument with other words that have no direct connection with the trigger word as negative sampling. Given a relation set E sel and the positive sample set P of a single cross-border ethnic cultural text s The specific formula is as follows:
[0087] E sel ={AGT,DATV,eCOO,eSUCC,EXP,FEAT,ROOT,SCO,TIME}
[0088] P s ={(u,v)|(u,v,r)∈G cb ,r∈E sel}
[0089] Step 3.1.3 In order to learn to distinguish between positive sample word pairs and negative sample word pairs, so that the model can learn better semantic representation of cross-border ethnic cultural events, the training target of the model is defined as the cross entropy loss that can correctly classify positive sample word pairs. The specific formula is as follows:
[0090]
[0091] Among them, (t, a) represents a positive example word pair, x t ,x a Represents the vector representation of trigger word t and event argument a respectively, m t and m a They represent the number of negative samples obtained by randomly replacing trigger words and the number of negative samples obtained by randomly replacing event arguments, respectively. and Respectively represent the vector representation of randomly replacing trigger words or event arguments, and W represents the weight matrix. Then, the loss obtained for each sentence is added together to obtain the overall training goal of event semantic pre-training. The specific formula is as follows:
[0092]
[0093] Among them, θ represents the parameters that need to be trained, S batch Represents a batch of cross-border national cultural texts.
[0094] As a preferred embodiment of the present invention, the specific steps of Step 3.2 are:
[0095] Step 3.2.1: For cross-border ethnic cultural events with complex event element relationships, using graph structures to model them can clearly represent the relationships between event elements. First, given a cross-border ethnic cultural event semantic structure graph g, its encoding can be expressed as the following formula:
[0096] x g =GNN(g,{x w})
[0097] Among them, x w is the node representation initialized by the text encoder. This node initialization method can implicitly align the semantic space of the event semantic representation and the structure representation in the model of this chapter, so that both can better represent cross-border ethnic cultural texts.
[0098] Step3.2.2, given a batch of m event structure graphs {g1,g2,...g m}, where each event structure graph corresponds to a sentence in the unsupervised corpus. For the i-th subgraph g i , randomly select two subgraphs from its subgraph and As positive examples, we randomly sampled subgraphs from other event structure graphs in the batch as negative examples. The subgraph pair consisting of two subgraphs G1-S1 and G1-S2 from the same mother graph G1 was used as a positive example; the subgraph pair consisting of other subgraphs G2-S1 and G2-S2 from the same batch and subgraph G1-S1 was used as a negative example, as shown in the attached figure. Figure 1 Similar to the contrast pre-training of event semantics, the present invention uses a graph encoder to train the subgraph g sub Encoding, then the graph compares the pre-trained loss function L gra It is defined as follows:
[0099]
[0100] Among them, 1 [j≠2i-1] ∈{0,1} is a discriminant function, which takes the value of 1 when j≠2i-1; θ is the parameter that needs to be trained.
[0101] As a preferred embodiment of the present invention, the specific steps of Step 3.3 are:
[0102] Step 3.3.1. Input the text into the pre-trained text encoder to obtain the semantic encoding representation of the text {x w}, and use the Dynamic Multi-pooling (DM) method to integrate it into a unified representation. The specific formula is as follows:
[0103] X s =DM({x w})
[0104] Step 3.3.2: Select the candidate trigger word or event argument node from the event structure graph, and input the subgraph consisting of the candidate trigger word or event argument node and the node with a direct edge between them into the event structure graph encoder to obtain the structural encoding representation X of the text. g The specific formula is as follows:
[0105] X g =Encoder g (g t′,a′ )
[0106] Step 3.3.3, concatenate the semantic coding representation and structural coding representation of the text as the coding representation of the cross-border ethnic cultural event text, and then send it into a multi-layer fully connected neural network, use softmax for normalization, and get the final classification result.
[0107] As a preferred embodiment of the present invention, the specific steps of Step 4 are:
[0108] Step 4.1. Save the trained model as a ".pth" file and deploy it to the local server. Use Sanic technology to convert the model into an application interface.
[0109] Step 4.2: Call the application interface of the model deployed to the local server on the Web side, directly call the model through the web page, and output the retrieved results to the front-end interface for display.
[0110] The present invention also provides a device for extracting cross-border ethnic cultural events based on event pre-training, including a module for executing the method provided above.
[0111] In order to verify the effectiveness of the cross-border ethnic cultural event extraction method based on event pre-training proposed in this invention, comparative experiments and ablation experiments were designed.
[0112] In order to illustrate the effect of the present invention, the present invention conducted the following comparative experiment, and the experimental data used are as follows. The artificially constructed data set used defines six event types and ten event argument types, with a total of 15,000 data. The data set is divided into a training set of 10,500 items, with 18,532 events; a validation set of 3,000 items, with 5,295 events; and a training set of 1,500 items, with 2,624 events.
[0113] In order to illustrate the effect of the present invention, the present invention conducted the following comparative experiments: In order to check the performance of the proposed model, the evaluation indicators used were precision, recall and F1 value to calculate the model effect.
[0114]
[0115] Where T is the number of event elements that are correctly identified and classified, F is the number of event elements with incorrectly identified entity labels, and F un is the number of unrecognized event elements. The effect of cross-border ethnic culture extraction is related to the F1 value. The higher the F1 value result, the better the effect of cross-border ethnic culture event extraction.
[0116] Experiment 1: Comparison of experimental results of different methods
[0117] As shown in Table 2, the method of the present invention is compared with DMCNN, DMBERT, EEQA and Text2Event methods under the condition of using event pre-training. DMCNN and DMBERT methods are based on the idea of dynamic multi-pooling, and extract event information from text through multiple channels to improve the effect of event extraction. However, for vertical fields such as cross-border ethnic culture, these methods have poor representation capabilities for cross-border ethnic cultural field information, resulting in poor model effects. The method of the present invention can better represent the field event information in the text through event pre-training, and the F1 value is improved by 6.85% and 2.52% respectively compared with it; the EEQA method is based on the idea of question answering, and constructs corresponding questions for event trigger words and event elements, and completes the event extraction task by extracting answers to questions from the text. However, there are some unique domain entities in the field of cross-border ethnic culture. In the absence of domain information, it is difficult for EEQA to extract the correct answer. Compared with it, the method of the present invention can better segment the cross-border ethnic cultural event text by using the domain dictionary in the model, and effectively represents the event information in the text through event pre-training. The experimental results show that the F1 value is improved by 1.74%. The Text2Event method analyzes the event text, converts the structured event information into a sequence structure, and extracts the event information from the text through an end-to-end generation model. However, the relationship between the elements of cross-border ethnic cultural events is relatively complex, which makes it difficult for the Text2Event model to generate correct event elements. In comparison, the method of the present invention has a better representation of the structural information of cross-border ethnic cultural events by incorporating event structure graph information. Experimental results show that the F1 value is improved by 1.34%.
[0118] Table 2 Comparative experimental results
[0119]
[0120] Experiment 2: The impact of different event pre-training on experimental results
[0121] The method of the present invention mainly incorporates the information of event pre-training. Therefore, ablation experiments are designed respectively without using event pre-training information, only using event semantic pre-training, and only using event structure pre-training to prove the effectiveness of event pre-training.
[0122] Table 3. Ablation experiment results
[0123]
[0124] As shown in Table 3, it can be seen from the experimental results that when the model directly uses the existing pre-trained language model, the existing model fails to learn enough cross-border ethnic cultural event information, resulting in poor event extraction. When event semantic pre-training or event structure pre-training is added to the model separately, the effect of the model is significantly improved, and the F1 value is increased by 4.8% and 6.04% respectively. Event pre-training can improve the model's ability to represent cross-border ethnic cultural texts, thereby improving the effect of event extraction. Integrating the two event pre-training methods into the model, the F1 value is increased by 8.1% compared to the baseline model. The event pre-training method can effectively improve the effect of event extraction, verifying the effectiveness of the method of the present invention.
[0125] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. A method for extracting cross-border ethnic cultural events based on event pre-training, characterized by: The specific steps of the method are as follows: Step 1: Construction of a dataset for extracting cross-border ethnic cultural events: Collect network resources to construct relevant data on cross-border ethnic groups, process the data, and finally construct a dataset for extracting cross-border ethnic cultural events; Step 2: Construction of cross-border ethnic cultural event structure diagram: Use semantic dependency analysis to construct the semantic structure diagram of the event, and alleviate the problem of difficulty in classifying event elements by integrating the event structure information into the display; Step 3: Construction of a cross-border ethnic cultural event extraction model based on event pre-training: By explicitly incorporating event structure information into the representation of cross-border ethnic cultural events, and due to the scarcity of labeled data, pre-training text encoders and event structure graph encoders based on contrastive learning, the representation of cross-border ethnic cultural events is further enhanced to alleviate the problem of poor extraction of cross-border ethnic cultural events; Step 4. Extraction of cross-border ethnic cultural events: Save the trained event extraction model, encapsulate it and deploy it on the server, establish an API to allow other applications to interact with the model, and realize the online extraction of cross-border ethnic cultural events.
2. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized in that: The specific steps of Step 1 are: Step 1.1, obtain relevant data on cross-border ethnic groups through Wikipedia, Twitter, and Kunong Lisu Nationality Network; Step 1.2, by analyzing the obtained cross-border ethnic cultural text data, cross-border ethnic cultural events are defined as six types of events: cultural and artistic events, religious sacrificial events, festival celebration events, historical legend events, food preparation events and NONE events; according to the content described in the text, they are classified into one of these six types of events; Step 1.
3. Manually use the event annotation tool publicly available at Stanford University to annotate the cross-border ethnic cultural event extraction dataset. First, import the cross-border ethnic cultural text data into the annotation tool to obtain the entity recognition data and part-of-speech analysis of this sentence. According to these part-of-speech analysis and named entity recognition results, annotate the sentence with event information. Annotate the verbs or nouns in the text that may become trigger words as trigger words. The entities related to the events corresponding to the trigger words are annotated as event arguments, and the roles of the corresponding event arguments are annotated with reference to the entity types; event information that is not recognized or identified incorrectly by the annotation tools is annotated through manual review; finally, a high-quality cross-border ethnic cultural event extraction dataset is annotated through manual annotation.
3. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized in that: The specific steps of Step 2 are: Step 2.1, automatically construct a semantic dependency analysis graph of cross-border ethnic cultural event texts through semantic dependency analysis tools; Step 2.2, select the edges and nodes connected between the edges that meet the cross-border ethnic cultural event relationship from the obtained semantic dependency analysis graph, and construct a cross-border ethnic cultural event structure graph; the process of obtaining the cross-border ethnic cultural event structure graph can be abstracted as: a cross-border ethnic cultural text, after semantic dependency analysis, obtains its event structure graph G cb ; The specific formula is as follows: G cb ={u,v,e|(u,v)∈V cb ×V cb ,e∈E sel } Among them, u, v represent vocabulary nodes, e represents the edge between nodes, V cb Represents a word node set, E sel Represents the selected set of relations.
4. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized in that: The specific steps of Step 3 are: Step 3.1, select the pre-trained language model as the text encoder of cross-border ethnic cultural events, and pre-train the text encoder based on the contrastive learning method to enhance the semantic representation of cross-border ethnic cultural events; Step 3.2, select the graph isomorphism neural network as the cross-border ethnic cultural event structure graph encoder, and pre-train the event structure graph encoder based on the contrastive learning method to enhance the structural representation of cross-border ethnic cultural events; Step 3.3, use the text encoder and event structure graph encoder together as encoders for cross-border ethnic cultural event extraction, and integrate the event text encoding representation and event structure encoding representation to extract cross-border ethnic cultural events.
5. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized in that: The specific steps of Step 3.1 are: Step 3.1.1, using the cross-border ethnic culture dictionary constructed in the previous work as an aid to better segment the cross-border ethnic culture text; given a cross-border ethnic culture text containing n words s = {w1, w2, ..., w n }, and combine with the domain dictionary to get {w1,w2,...,w n ,v1…vm}, where w represents a high-frequency word and v represents a domain vocabulary; the high-frequency word w is sent to the pre-trained language model (RoBERTa), and the hidden layer vector x of the last layer is selected w As a representation of cross-border ethnic cultural texts, low-frequency words are randomly initialized for word embedding; the specific formula is as follows: x w =RoBERTa(w i ) x d =random_word2vec(v i ) Step 3.1.2: For the semantic pre-training of cross-border ethnic cultural events, a "trigger word-event argument" pair is constructed as positive sampling, and a negative sampling is performed by randomly replacing the trigger word with other words that have no direct connection with the event argument or randomly replacing the event argument with other words that have no direct connection with the trigger word. Given a relation set E sel and the positive sample set P of a single cross-border ethnic cultural text s ; The specific formula is as follows: E sel ={AGT,DATV,eCOO,eSUCC,EXP,FEAT,ROOT,SCO,TIME} P s ={(u,v)|(u,v,r)∈G cb ,r∈E sel } Step 3.1.3 defines the training objective of the model as the cross entropy loss that can correctly classify positive sample word pairs; the specific formula is as follows: Among them, (t, a) represents a positive example word pair, x t ,x a Represents the vector representation of trigger word t and event argument a respectively, m t and m a They represent the number of negative samples obtained by randomly replacing trigger words and the number of negative samples obtained by randomly replacing event arguments, respectively. and Respectively represent the vector representation of randomly replacing the trigger word or event argument, and W represents the weight matrix; then, the loss obtained for each sentence is added to obtain the overall training goal of event semantic pre-training;; The specific formula is as follows: Among them, θ represents the parameters that need to be trained, S batch Represents a batch of cross-border national cultural texts.
6. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized by: The specific steps of Step 3.2 are: Step 3.2.
1. For cross-border ethnic cultural events with complex event element relationships, using graph structures to model them can clearly represent the relationships between event elements. First, given a cross-border ethnic cultural event semantic structure graph g, its encoding can be expressed as the following formula: x g =GNN(g,{x w }) Among them, x w is the node representation initialized by the text encoder; Step3.2.2, given a batch of m event structure graphs {g1,g2,...g m }, where each event structure graph corresponds to a sentence in the unsupervised corpus; for the i-th subgraph g i , randomly select two subgraphs from its subgraph and As positive examples, we randomly sample subgraphs from other event structure graphs in the batch as negative examples. Similar to the contrastive pre-training of event semantics, we use a graph encoder to train the subgraph g sub Encoding, then the graph compares the pre-trained loss function L gra It is defined as follows: Among them, 1 [j≠2i-1] ∈{0,1} is a discriminant function, which takes the value of 1 when j≠2i-1; θ is the parameter that needs to be trained.
7. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized in that: The specific steps of Step 3.3 are: Step 3.3.
1. Input the text into the pre-trained text encoder to obtain the semantic encoding representation of the text {x w }, and use the dynamic multi-pooling method to integrate it into a unified representation; the specific formula is as follows: X s =DM({x w }) Step 3.3.2: Select the candidate trigger word or event argument node from the event structure graph, and input the subgraph consisting of the candidate trigger word or event argument node and the node with a direct edge between them into the event structure graph encoder to obtain the structural encoding representation X of the text. g ; The specific formula is as follows: X g =Encoder g (g t′,a′ ) Step 3.3.3, concatenate the semantic coding representation and structural coding representation of the text as the coding representation of the cross-border ethnic cultural event text, and then send it into a multi-layer fully connected neural network, use softmax for normalization, and get the final classification result.
8. The method for extracting cross-border ethnic cultural events based on event pre-training according to claim 1 is characterized by: The specific steps of Step 4 are: Step 4.
1. Save the trained model as a ".pth" file and deploy it to the local server. Use Sanic technology to convert the model into an application interface. Step 4.2: Call the application interface of the model deployed to the local server on the Web side, directly call the model through the web page, and output the retrieved results to the front-end interface for display.
9. A cross-border ethnic cultural event extraction device based on event pre-training, characterized in that: The method comprises a module for executing the method according to any one of claims 1 to 8.