Multi-scale comparison pre-training method for small sample event detection

Abstract

The invention relates to the technical field of small sample event detection, and provides a multi-scale comparison pre-training method for small sample event detection. The method aims at solving the problems that an existing event detection method depends on a large amount of annotated data, and the new event type detection effect is poor in a low-resource environment. According to the main scheme, the method comprises the steps that an event type label concept dictionary is constructed, an AMR structure of a sentence set is obtained through an automatic AMR analyzer, and a trigger semantic enhancement encoder, a sentence semantic information encoder and an event type information encoder are created. Through comparative learning, encoders are respectively constructed on two dimensions of sentence-trigger level information and sentence-event type level information, and semantic representations of input text information and event type information are enriched, so that the performance of an event detection model in a low-resource scene is improved. The method is used for improving the accuracy and robustness of small sample event detection.

Description

Technical Field

[0001] The present invention relates to the technical field of small sample event detection, and in particular to a multi-scale comparison pre-training method for small sample event detection. Background Art

[0002] The task of event detection aims to identify and classify specific events from unstructured text, help convert text information into structured data, and support further analysis and application. However, event detection methods often rely on a large amount of annotated data for supervised training, which poses a challenge to the detection of unseen or rare events. Since these event types appear infrequently or not at all in the training data, existing models are usually unable to effectively identify them. Therefore, it is particularly important to study small sample event detection tasks in order to improve the generalization and detection capabilities of the model in data-scarce scenarios.

[0003] Furthermore, in the event detection task, the existing methods almost follow the supervised learning paradigm and rely on large-scale manually annotated datasets. In the case of insufficient resources, the existing joint model cannot identify new event types with only a few samples, that is, small sample event detection. In modern deep learning models, the use of pre-training brings significant advantages in data-limited tasks. However, in the event detection task, there are currently few studies on the use of pre-training methods and most methods attempt to solve the problem of identifying new event types with a small number of samples in the form of recognition and then classification, but ignore the impact of the contextual context within the sentence on the trigger semantics, as well as the intrinsic relationship between the semantics of the sentence and the event type, thus suffering from error propagation. Therefore, this patent proposes a multi-scale contrastive pre-training method for small sample event detection, and constructs two different contrastive pre-training encoders in the two dimensions of sentence-trigger level information and sentence-event type level information, respectively, to enrich the semantic representation of input text information and event type information, supplement the label description of the event type, and enrich the semantics of the event type information. Combining the above two types of information in fine-tuning can significantly improve the performance of the event detection model in low-resource scenarios. Summary of the invention

[0004] The present invention aims to solve the problems in existing research methods, that is, event detection is heavily dependent on trigger recognition, the intrinsic semantic relationship between sentences and event types, and the poor detection effect of new event types in low-resource environments due to ignoring the influence of the contextual context within the sentence on the trigger semantics. Therefore, the present invention is based on a public knowledge base. For information within a sentence, the text information (sentences, trigger information) is input into the trigger semantic enhancement encoder, and contrastive learning is used to make the trigger obtain different representations according to the corresponding arguments in the context, thereby reducing the dependence of event detection on trigger recognition, improving the utilization of context, and better modeling local event semantics. For information between sentences, the new text representation and the reconstructed event type representation are mapped to the same vector representation space respectively through a dual encoder architecture: contrastive learning is used to promote two-way learning between the text information encoder and the event type information encoder to generate a more robust relationship representation, thereby improving the performance of small sample event detection, thereby solving the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A multi-scale contrast pre-training method for small sample event detection, the event detection method has the following specific steps:

[0007] Step S10: construct an event type label concept dictionary based on the public knowledge base and the large language model, and construct an associated knowledge base together with the training sentences;

[0008] Step S20: using an automatic AMR parser to obtain the AMR structure of the sentence set;

[0009] Step S30: Create a trigger semantic enhancement encoder to generate new text representations using intra-sentence contrast learning;

[0010] Step S40: creating a sentence semantic information encoder based on the new text representation obtained in S30 to generate a sentence semantic representation;

[0011] Step S50: creating an event type semantic information encoder to generate an event type semantic representation;

[0012] Step S60: performing comparative learning on the sentence semantic representation and event type semantic representation obtained in S40 and S50 to establish a pre-trained language representation model;

[0013] Step S70: Utilize the sentence semantic information encoder and event type semantic information encoder pre-trained in step S40 and step S50 to encode the sentence and event type information respectively, and assist the prototype network in performing small sample event detection.

[0014] Preferably, the specific steps of step S10 are as follows:

[0015] Step S101, obtaining an original sentence set sentence from a public knowledge base used for pre-training of small sample event detection set And the corresponding event type label set label set . Sentence set set The original representation of each sentence in is S I =[t1, t2, …, t n ], 0<I<N, n is the number of words in the sentence, and N is the number of sentences in the sentence set;

[0016] Step S102, obtaining the event type label set label in step S101 on the large language model set The event type label and label description corresponding to each event type in the event type label are concatenated with the corresponding event type label description to construct the event type label concept dictionary type set , where the original representation of each event type is T I =[label I ,des I ]=[l1,l2,…,l t ], 0<I<N, t is the number of words in the event type information, and N is the number of event types in the event type label concept dictionary. The sentence set sentence in step S101 set Dictionary of concepts with event type tags type set Jointly build a pre-trained associative knowledge base.

[0017] Preferably, the specific steps of step S20 are as follows:

[0018] Step S201: The original sentence set is known set , according to the automatic AMR parser, we get the AMR graph set g s =(V s , E s ), V s is the node set after word merging, E s Represents an edge set. Is a collection of defined semantic relationship types.

[0019] Preferably, the specific steps of step S30 are as follows:

[0020] Step S301: The original sentence set sentence is known set , for the Kth sentence, let Know Represents sentence S K The set of positive trigger word-argument pairs is obtained by sampling m t Negative trigger Combined with the positive element a to construct m t negative sample pairs. And negative trigger cannot have a directed ARG, time, or position edge with argument a. Similarly, by randomly sampling m a Negative parameters To construct m a More negative sample pairs. In order to learn to distinguish positive trigger-argument pairs from negative trigger-argument pairs and model the semantics of intra-sentence events, the training objective of the positive trigger-argument pair (t, a) is defined as the cross entropy loss of correctly classifying the positive trigger-argument pair:

[0021]

[0022] Among them, m t 、m a is a hyperparameter for negative sampling, and W is a trainable matrix for learning similarity metrics. set The overall loss function of the N sentences in is and the new text representation x t，a .

[0023] Preferably, the specific steps of step S40 are as follows:

[0024] Step S401: the original sentence set sentence is known set The representation of the I-th sentence in is S I =[t1, t2, …, t r ,…,t n ], t r As the trigger word of the sentence, insert a special token [[unused0]] in the original representation of the sentence, t r , [unused1]] represent the start and end positions of the trigger in the sentence, forming a new sentence representation

[0025] Step S402: construct a sentence semantic information encoder Φ based on Bert s , the above sentence can be expressed as Input to the semantic information encoder Φ s Generate sentence vector representation in where h clsis the global representation of the sentence, and f and g are the corresponding positions of the inserted special tags [unused0] and [unused1]. Combined with the text representation obtained by S30, the final sentence semantic representation is obtained.

[0026] Preferably, the specific steps of step S50 are as follows:

[0027] Step S501: a set of reconstruction event types is known. set The Jth sentence in is represented by T J =[e1, e2, …, e x , e x+1 ,…,e m ],[e1,e2,…,e x ] is the name of the event type, [e x+1 ,…,e m ] is a detailed description of the event type. A special marker [SEP] is inserted into the original representation of the event type to separate the event type name and the event type description to form a new event type representation.

[0028] Step S502: construct an event type information encoder Φ based on Bert t , known original event type concept dictionary type set The J-th event type in is represented as Will Input to the event type information encoder Φ t Generate event type vector representation in where e cls is the global representation of the event type vector, and the semantic representation of the event type corresponding to the i-th event type label is obtained in It means to concatenate two vectors, V mean is a vector of event types Obtained by average pooling.

[0029] Preferably, the specific steps of step S60 are as follows:

[0030] Step S601, step S402 and step S502 obtain N sentence semantic representations and M event types. The goal is to predict the actual (sentence, label) from N×M possible combinations. During the training process, the goal is to maximize the cosine similarity between the embeddings of the N correct pairs and minimize the cosine similarity of the embeddings belonging to the N×MN wrong pairs. In order to achieve effective comparison, a two-way comparison target is adopted, which contains two components: sentence-label comparison learning (SCL) and label-sentence comparison learning (SCLI). This symmetric comparison target ensures that the framework captures the intrinsic relationship between sentences and labels.

[0031] Step S602, sentence-label contrast learning is based on each individual sentence S i As anchor points, positive and negative samples are extracted from the event type label set. Specifically, for the positive pair (S i , l i ) in a given sentence S i , any label in the remaining pairs is the same as S i Form a negative pair, expressed as (S i , l j ), where 1≤j≤m-1, the loss function of the training process is expressed as follows:

[0032]

[0033] in is the sentence information representation obtained in step S202, is the event type information obtained in step S302, τ is a temperature parameter, sim represents a function for calculating cosine similarity, and exp() represents an exponential function with e as the base;

[0034] Step S603: label-sentence contrastive learning symmetrically uses the event type label as an anchor point and extracts positive and negative samples from the corresponding sentence set. i , set A includes all the labels l i The positive sentences are those with labels other than l i The loss function of the training process is expressed as follows:

[0035]

[0036] The final contrastive training loss is calculated as a combination of the two losses:

[0037] Step S604: To maintain the language comprehension ability of the encoder during pre-training back propagation, mask training is required for the input vector. The semantic information encoder Φ s and event type information encoder Φ tThe mask training loss functions are and The loss function of contrastive learning is added together to form the final loss of the pre-trained representation model:

[0038]

[0039] Preferably, the specific steps of step S70 are as follows:

[0040] Step S701, applying the small sample task N-way-K-shot setting, that is, dividing the original data set into two parts: a support set and a query set, the support set contains N event types, each event type has K instances, and the query set has M sentence instances, but there is no event type label corresponding to the sentence;

[0041] Step S702, based on the idea of ​​prototype network, small sample event detection is performed, through the semantic information encoder Φ of step S40 and step S50 s and event type information encoder Φ t Encode the sentence i and event type information j in the support set to obtain the sentence semantic representation and event type information representation Then the two representations are combined to obtain the prototype representation Then use the sentence semantic information encoder Φ s Encode the sentence instance q in the query set to obtain the sentence semantic representation

[0042] Step S703, calculate With different prototypes R final The cosine similarity is used to represent the distance between the query instance and the representation space of different class prototypes, and the event category corresponding to the prototype with the shortest distance is selected as the prediction result. The loss function used in this small sample event detection task is as follows:

[0043] L CE = -log(z y )

[0044]

[0045] Where y represents the event type label, z y Represents the semantic representation of the sentence to be predicted After calculation, the probability that the sentence belongs to event type y is expressed as The type of real event to which it belongs.

[0046] The beneficial effects of the present invention are:

[0047] The present invention aims to solve the problem that the existing research methods ignore the influence of the contextual context in the sentence on the trigger semantics and the intrinsic relationship between the semantics of the sentence and the event type, resulting in the existing methods being heavily dependent on trigger identification in event detection research, not seeing the difference between event types, and the detection performance being quite unbalanced, and the problem that the detection effect of new event types is poor due to the inability to obtain a large amount of annotated data in a low-resource environment. Therefore, based on the public knowledge base, the present invention inputs the text information (sentences, trigger information) into the trigger semantic enhancement encoder for the information in the sentence, and uses contrastive learning to enable the trigger to learn more semantic information in the sentence according to the context, rather than only the semantic information of the trigger itself, thereby reducing the dependence of event detection on trigger identification, improving the utilization of context, and better modeling local event semantics. For inter-sentence information, the new text representation and the reconstructed event type representation are mapped to the same vector representation space respectively through the dual encoder architecture: contrastive learning is used to promote the two-way learning between the text information encoder and the event type information encoder to generate a more robust relationship representation, thereby improving the performance of small sample event detection, thereby effectively solving the problems of high trigger dependence, weak semantic association, and poor detection effect of new event types in event detection, and achieving a more accurate and robust detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] See also Figure 1 , the present invention provides a technical solution:

[0051] The present invention provides the following technical solution: a multi-scale contrast pre-training method for small sample event detection. The specific steps are to use small sample event detection of news as an example. The specific steps of applying the small sample event detection method to the field of news event detection are as follows:

[0052] Step 1: Based on the public knowledge base and large language model, construct an event type label concept dictionary, and build an associated knowledge base together with the training sentences;

[0053] Step 1-1, obtain the original sentence set sentence from the public knowledge base used for pre-training of small sample event detectionset And the corresponding event type label set label set . Sentence set set The original representation of each sentence in is S I =[t1, t2, …, t n ], 0<I<N, n is the number of words in the sentence, and N is the number of sentences in the sentence set;

[0054] Step 1-2: Get the event type annotation set label in step 1-1 on the large language model set The event type label and label description corresponding to each event type in the event type label are concatenated with the corresponding event type label description to construct the event type label concept dictionary type set , where the original representation of each event type is T I =[label I ,des I ]=[l1,l2,…,l t ], 0<I<N, t is the number of words in the event type information, and N is the number of event types in the event type label concept dictionary. The sentence set sentence in step S101 set Dictionary of concepts with event type tags type set Jointly build a pre-trained associative knowledge base;

[0055] Step 2: Use the automatic AMR parser to obtain the AMR structure of the sentence set;

[0056] Step 2-1: Known original sentence set sentence set , according to the automatic AMR parser, we get the AMR graph set g s =(V s , E s ), V s is the node set after word merging, E s Represents an edge set. is a set of defined semantic relationship types;

[0057] Step 3: Create a trigger semantic enhancement encoder to generate new text representations using intra-sentence contrastive learning;

[0058] Step 3-1: Known original sentence set sentence set , for the Kth sentence, let and Represents sentence S K The set of positive trigger word-argument pairs is obtained by sampling mt Negative trigger Combined with the positive element a to construct m t negative sample pairs. And negative trigger cannot have a directed ARG, time, or position edge with argument a. Similarly, by randomly sampling m a Negative parameters To construct m a More negative sample pairs. In order to learn to distinguish positive trigger-argument pairs from negative trigger-argument pairs and model the semantics of intra-sentence events, the training objective of the positive trigger-argument pair (t, a) is defined as the cross entropy loss of correctly classifying the positive trigger-argument pair:

[0059]

[0060] Among them, m t 、m a is a hyperparameter for negative sampling, and W is a trainable matrix for learning similarity metrics. set The overall loss function of the N sentences in is:

[0061] and the new text representation x t，a .

[0062] Step 4: Based on the new text representation obtained in step 3, create a sentence semantic information encoder to generate a sentence semantic representation;

[0063] Step 4-1, the original sentence set sentence is known set The representation of the I-th sentence in is S I =[t1, t2, …, t r ,·…,t n ], t r As the trigger word of the sentence, insert a special token [[unused0]] in the original representation of the sentence, t r , [unused1]] represent the start and end positions of the trigger in the sentence, forming a new sentence representation

[0064] Step 4-2: Construct a sentence semantic information encoder Φ based on Bert s , the above sentence can be expressed as Input to the semantic information encoder Φ s Generate sentence vector representation in where h clsis the global representation of the sentence, and f and g are the corresponding positions of the inserted special tags [unused0] and [unused1]. Combined with the text representation obtained by S30, the final sentence semantic representation is obtained.

[0065] Step 5: Label the event types in the pre-training dataset set , create a Bert-based event type information encoder to generate event type representation;

[0066] Step 5-1, the reconstruction event type set type is known set The Jth sentence in is represented by T J =[e1, e2, …, e x , e x+1 ,…,e m ],[e1,e2,…,e x ] is the name of the event type, [e x+1 ,…,e m ] is a detailed description of the event type. A special marker [SEP] is inserted into the original representation of the event type to separate the event type name and the event type description to form a new event type representation.

[0068] Step 5-2: Construct the event type information encoder Φ based on Bert t , known original event type concept dictionary type set The J-th event type in is represented as Will Input to the event type information encoder Φ t Generate event type vector representation in where e cls is the global representation of the event type vector, and the semantic representation of the event type corresponding to the i-th event type label is obtained in It means concatenating two vectors;

[0069] Step 6: Compare and study the sentence semantic representation and event type semantic representation obtained in steps 4 and 5 to establish a pre-trained language representation model;

[0070] Step 6-1, step 4-2 and step 5-2 obtain N sentence semantic representations and M event types. The goal is to predict the actual (sentence, label) from N×M possible combinations. During the training process, the goal is to maximize the cosine similarity between the embeddings of the N correct pairs, while minimizing the cosine similarity of the embeddings belonging to the N×MN wrong pairs. In order to achieve effective comparison, a two-way comparison target is adopted, which contains two components: sentence-label comparison learning (SCL) and label-sentence comparison learning (SCLI). This symmetric comparison target ensures that the framework captures the intrinsic relationship between sentences and labels;

[0071] Step 6-2, sentence-label contrastive learning takes each individual sentence S i As anchor points, positive and negative samples are extracted from the event type label set. Specifically, for the positive pair (S i , l i ) in a given sentence S i , any label in the remaining pairs is the same as S i Form a negative pair, expressed as (S i , l j ), where 1≤j≤m-1, the loss function of the training process is expressed as follows:

[0072]

[0073] in is the sentence information representation obtained in step 2-2, is the event type information obtained in step 3-2, τ is the temperature parameter, sim represents the function for calculating cosine similarity, and exp() represents the exponential function with base e;

[0074] Step 6-3, label-sentence contrastive learning symmetrically uses the event type label as an anchor and extracts positive and negative samples from the corresponding sentence set. For a given label li, set A includes all labels with l i The positive sentences are those with labels other than l i The loss function of the training process is expressed as follows:

[0075]

[0076] The final contrastive training loss is calculated as a combination of the two losses:

[0077] Step 6-4: To maintain the language understanding ability of the encoder during pre-training back propagation, mask training of the input vector is required, and the semantic information encoder Φ s and event type information encoder Φ t The mask training loss functions are and The loss function of contrastive learning is added together to form the final loss of the pre-trained representation model:

[0078]

[0079] Step 7: Use the two pre-trained encoders obtained in Step 4 and Step 5 to encode sentence and event type information respectively, and assist the prototype network in performing small sample event detection;

[0080] Step 7-1, apply the small sample task N-way-K-shot setting, that is, divide the original data set into two parts: support set and query set. The support set contains N event types, each event type has K instances, and the query set has M sentence instances, but there is no event type label corresponding to the sentence;

[0081] Step 7-2: Detect small sample events based on the idea of ​​prototype network, through the semantic information encoder Φ of step 4 and step 5 s and event type information encoder Φ t Encode the sentence i and event type information j in the support set to obtain the sentence semantic representation and event type information representation Then the two representations are combined to obtain the prototype representation Then use the sentence semantic information encoder Φ s Encode the sentence instance q in the query set to obtain the sentence semantic representation

[0082] Step 7-3, Calculate With different prototypes R final The cosine similarity is used to represent the distance between the query instance and the representation space of different class prototypes, and the event category corresponding to the prototype with the shortest distance is selected as the prediction result. The loss function used in this small sample event detection task is as follows:

[0083] L CE = -log(z y )

[0084]

[0085] Where y represents the event type label, z y Represents the semantic representation of the sentence to be predicted After calculation, the probability that the sentence belongs to event type y is expressed as The type of real event to which it belongs.

[0086] The multi-scale contrastive pre-training method for abstract semantic enhancement for small sample event detection includes a trigger semantic enhancement encoder module, a trigger-argument pair local contrastive learning module, a sentence semantic information representation encoder module, an event type information representation encoder, and a sentence information and event type information global contrastive learning module. Specifically:

[0087] A local contrast learning mechanism based on the information representation of triggers and corresponding arguments is established, and the encoder is enhanced by trigger semantics to generate new sentence information representation for the original sentence set.

[0088] Create a sentence semantic information representation encoder to generate sentence semantic information representation based on semantics and trigger positions;

[0089] Create an event type information representation encoder to generate an event type information representation based on the event type label and its description;

[0090] A global contrastive learning mechanism based on sentence semantic information representation and event type information representation is established to improve the similarity between the sentences generated by the above two encoders and their corresponding event type representations, so that the encoders can learn the potential relationship between the two, thereby promoting mutual learning between encoders, and then generating better semantic representations and improving small sample event detection performance.

[0091] The method of the present invention takes the small sample event detection method as the research object, constructs a sentence set and a label dictionary based on a public knowledge base, and uses a large language model to generate event type descriptions. Then, the AMR structure of the sentence set is obtained using an automatic AMR parser. For information within a sentence, the text information (sentences, trigger information) is input into the trigger semantic enhancement encoder, and contrastive learning is used to make the trigger obtain different representations according to the context corresponding arguments, thereby reducing the dependence of event detection on trigger recognition, improving the utilization rate of context, and better modeling local event semantics. For information between sentences, the new text representation and the reconstructed event type representation are respectively mapped to the same vector representation space through a dual encoder architecture: contrastive learning is used to promote two-way learning between the text information encoder and the event type information encoder, thereby generating a better global semantic representation and improving the event detection performance based on a small sample.

[0092] The present invention has the following characteristics because it adopts the above technical solution:

[0093] 1. Construction of event type label concept dictionary: The present invention first constructs an event type label concept dictionary through a public knowledge base and a large language model. This dictionary not only contains the labels of event types, but also includes a detailed description of each event type. This construction method enriches the semantic representation of event type information and helps the model better understand and identify new event types in a low-resource environment. Compared with the traditional method that relies only on event type labels, this method has a more comprehensive and in-depth understanding of the meaning of event types.

[0094] 2. Trigger semantic enhancement encoder: In the processing of information within a sentence, the present invention adopts a trigger semantic enhancement encoder. The encoder uses contrastive learning to make the trigger have different representations according to the corresponding arguments in the context. This method reduces the dependence of event detection on trigger recognition, improves the utilization of context, and better models local event semantics. The innovation of this method is that it not only focuses on the semantic information of the trigger itself, but also pays more attention to the relationship between the trigger and the context, thereby improving the accuracy of event detection.

[0095] 3. Sentence semantic information encoder and event type information encoder: The present invention also creates a sentence semantic information encoder and an event type information encoder. These two encoders generate sentence semantic representation and event type semantic representation respectively. The design of this dual encoder architecture enables sentence information and event type information to be comparatively learned in the same vector representation space, thereby generating a better global semantic representation and improving small sample event detection performance.

[0096] 4. Implementation of small sample event detection: When implementing small sample event detection, the present invention adopts the idea of ​​prototype network. The pre-trained encoder encodes the sentence and event type information respectively, calculates the cosine similarity between the sentence and different event types, and selects the most similar event type as the prediction result. This method performs well on small sample data sets and improves the accuracy and robustness of event detection.

[0097] 5. Comparison with the prior art: Compared with the prior art, the advantage of the present invention is that it not only utilizes the training sentences, but also supplements the detailed description of the event type, thereby enriching the semantic representation of the event type information. In addition, through contrastive learning and dual encoder architecture, the present method can more accurately model the semantic information within and between sentences, thereby improving the performance of small sample event detection. Especially in low-resource environments, the present invention can more effectively identify and classify new event types and has high practical value.

[0098] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-scale contrast pre-training method for small sample event detection, characterized in that: The following steps are involved: step S10: Based on the public knowledge base and large language model, construct a sentence set and event type label concept dictionary, and build an associated knowledge base together with the training sentences; Step S20: using an automatic AMR parser to obtain the AMR structure of the sentence set; Step S30: Create a trigger semantic enhancement encoder and construct local contrast learning to generate new text representation; Step S40: New text representation x obtained in step S30 t，a , create a sentence semantic information encoder to generate sentence semantic representation; Step S50: creating an event type semantic information encoder to generate an event type semantic representation; Step S60: Compare and study the sentence semantic representation and event type semantic representation obtained in step S40 and step S50 to establish a pre-trained language representation model; Step S70: Utilize the sentence semantic information encoder and event type semantic information encoder pre-trained in step S40 and step S50 to encode the sentence and event type information respectively, and assist the prototype network to perform event detection based on small samples.

2. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: The specific steps of step S10 are as follows: Step S101, obtaining an original sentence set sentence from a public knowledge base used for pre-training of small sample event detection set And the corresponding event type label set label set ; Step S102, obtaining the event type label set label in step S101 from the public knowledge base and the large language model set The event type label and label description corresponding to each event type in the event type are concatenated with the corresponding event type label description to construct the event type label concept dictionary type set , and the sentence set sentence in step S101 set Jointly build a pre-trained associative knowledge base.

3. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: The specific steps of step S20 are as follows: Step S201: The original sentence set is known set , according to the automatic AMR parser, we get the AMR graph set g s =(V s , E s ), V s is the node set after word merging, E s represents the edge set, is a set of defined semantic relationship types, where (u, v, r) represents an edge, u and v are two nodes, which are the two endpoints of the edge, and r is the semantic relationship type represented by the edge, which defines the semantic relationship between u and v.

4. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: Step S30 includes: Step S301: The original sentence set sentence is known set , for the Kth sentence S K ,make and Represents sentence S K The set of positive trigger-argument pairs, where It represents an existential quantifier, meaning "there is at least one". Represents sentence S K The edge set in the AMR graph of ; By sampling m t Negative trigger Combined with the positive element a to construct m t negative sample pairs, and negative triggers cannot have a directed ARG, time, or position edge with argument a; By randomly sampling m a Negative parameters To construct m a In order to learn to distinguish positive trigger-argument pairs from negative trigger-argument pairs and model the semantics of intra-sentence events, the training objective of the positive trigger-argument pair (t, a) is defined as the cross entropy loss of correctly classifying the positive trigger-argument pair: in Represents sentence S K The node set after the words are merged, The feature vector x representing the trigger word t t The transpose of Represents the i-th negative trigger word The eigenvector of The transpose of x a represents the eigenvector of argument a, represents the jth negative argument The eigenvector of t 、m a is a hyperparameter of negative sampling, W is a trainable matrix for learning similarity metrics, for the sentence set sentence set The overall loss function of the N sentences in is And the new semantically enhanced trigger text representation x t，a .

5. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: Step S40 includes: Step S401: the original sentence set sentence is known set The representation of the I-th sentence in is S I =[t1, t2, …, t r ,…,t n ], t r As the trigger word of the sentence, insert a special token [[unused0]] in the original representation of the sentence, t r , [unused1]] represent the start and end positions of the trigger in the sentence, forming a new sentence representation Step S402: construct a sentence semantic information encoder Φ based on Bert s , the sentence is expressed as Input to the semantic information encoder Φ s Generate sentence vector representation in , where h cls is the global representation of the sentence, h n+2 Representation is the representation of each word, h f With h g The corresponding positions of the inserted special tags [unused0] and [unused1] are combined with the text representation obtained by S30 to obtain the final sentence semantic representation 6. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: Step S501: a set of reconstruction event types is known. set The representation of the Jth sentence in is: T J =[e1,e2,…,e x ,the x+1 ,…,the m ] [e1, e2, …, e x ] is the name of the event type, [e x+1 ,…,e m ] is a detailed description of the event type. A special marker [SEP] is inserted into the original representation of the event type to separate the event type name and the event type description to form a new event type representation: Step S502: construct an event type information encoder Φ based on Bert t , it is known that the Jth event type in the new event type representation is represented as Will Input to the event type information encoder Φ t Generate the final event type vector representation in where e cls is the global representation of the event type vector, and the semantic representation of the event type corresponding to the i-th event type label is obtained ,in It means concatenating two vectors.

7. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: The specific steps of step S60 are as follows: Step S601, step S402 and step S502 obtain N sentence semantic representations and M event type semantic representations. The goal is to predict the actual {sentence, label} from N×M possible combinations. During the training process, the goal is to maximize the cosine similarity between the embeddings of the N correct pairs and minimize the cosine similarity of the embeddings belonging to the N×MN wrong pairs. Sentence-label contrastive learning and label-sentence contrastive learning are used to ensure that the intrinsic relationship between sentences and labels is captured; Step S602, sentence-label contrast learning is based on each individual sentence S i As anchor points, positive and negative samples are extracted from the event type label set. Specifically, for the positive pair (S i , l i ) in a given sentence S i , any label in the remaining pairs is the same as S i Form a negative pair, expressed as (S i , lj), where 1≤j≤m-1, the loss function of the training process is expressed as follows: Among them l i Represents sentence S i The corresponding event type label, is the sentence semantic representation obtained in step S402, is the semantic representation of the event type obtained in step S502, τ is the temperature parameter, sim represents the function for calculating cosine similarity, exp() represents the exponential function with e as the base, and n represents the sentence S i The number of sentences in , m represents the number of event type labels; Step S603, label-sentence contrastive learning symmetrically uses the event type label as an anchor point and extracts positive and negative samples from the corresponding sentence set. For a given label l i , set A includes all the labels l i The positive sentences are those with labels other than l i The loss function of the training process is expressed as follows: in is the semantic representation of the event type obtained in step S502. The final contrastive training loss is calculated as a combination of two losses: Step S604: To maintain the language comprehension ability of the encoder during pre-training back propagation, mask training is required for the input vector. The semantic information encoder Φ s and event type information encoder Φ t The mask training loss functions are and The loss function of contrastive learning is added together to form the final loss of the pre-trained representation model:

8. The multi-scale contrast pre-training method for abstract semantic enhancement for small sample event detection according to claim 1, characterized in that: The specific steps of step S70 are as follows: Step S701, applying the small sample task N-way-K-shot setting, divides the original data set into two parts: a support set and a query set, wherein the support set contains N event types, each event type has K instances, and the query set contains M sentence instances, but there is no event type label corresponding to the sentence; Step S702, performing small sample event detection based on the prototype network, through the semantic information encoder Φ of step S20 and step S30 s and event type information encoder Φ t Encode the sentence i and event type information j in the support set to obtain the sentence semantic representation And event type information representation Then the two representations are combined to obtain the prototype representation Then use the sentence semantic information encoder Φ s Encode the sentence instance q in the query set to obtain the sentence semantic representation Step S703, calculate With different prototypes R final The cosine similarity is used to represent the distance between the query instance and the representation space of different class prototypes, and the event category corresponding to the prototype with the shortest distance is selected as the prediction result. The loss function used in this small sample event detection task is as follows: L CE =-log(z y ) Where y represents the event type label, z y Represents the semantic representation of the sentence to be predicted After calculation, the probability that the sentence belongs to event type y is expressed as The type of real event it belongs to, Represents sentence i and event type l based on the support set i The prototype representation of .

Images