Zero sample event detection method for identifying all classes based on minority classes

The representative prototype subset is extracted through the prototype network and STIC algorithm, and combined with the Sentence-Transformer and Transformer models, the problem of zero-sample event detection relying on auxiliary information is solved, and the efficiency and accuracy of event detection is improved, and it is suitable for emerging fields and fast Changeable environment.

CN119003776APending Publication Date: 2024-11-22SHANGHAI JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411141280.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing zero-sample event detection method relies on additional auxiliary information, resulting in high model complexity and low detection efficiency. Especially when facing unknown event types, it is difficult to effectively reduce the dependence on manual annotation and improve detection accuracy.

Method used

Using a zero-sample event detection method based on a few classes to identify all classes, extract representative prototype subsets through prototype networks and STIC algorithms, use the Sentence-Transformer model for encoding and Transformer classifier for training, reduce dependence on manual annotation and improve detection Efficiency and accuracy.

Benefits of technology

Effectively capture the similarities and differences between different event types, reduce the misjudgment rate and missed detection rate, is suitable for emerging fields and rapidly changing market environments, and improves the detection ability of unknown event types.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003776A_ABST
    Figure CN119003776A_ABST
Patent Text Reader

Abstract

The invention discloses a zero sample event detection method for identifying all classes based on minority classes, which relates to the field of natural language processing and comprises the following steps of: dividing a data set into a known event type and an unknown event type; encoding the original text of the known event type; obtaining a prototype set through the prototype network; extracting a prototype subset from the prototype set by adopting an STIC algorithm; calculating the similarity between the encoded data of the original text and the prototype subset, and generating a feature data set; encoding the feature data set; and inputting the encoded feature data set into a Transform classifier to carry out training and classification. According to the method, the prototype subsets are extracted by using the prototype network and the STIC algorithm, the similarity and the difference between different event types are effectively captured, the highly accurate classification capability can still be kept when the event types are unknown, the misjudgment rate and the omission ratio are greatly reduced, and meanwhile, due to the fact that the method does not depend on a large amount of marked sample data, the classification efficiency is greatly improved. And the requirement of zero sample event detection can be effectively met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a zero-sample event detection method for identifying all classes based on a minority class. Background Art

[0002] In the field of natural language processing, event detection is a classic task that aims to identify and extract event type information from text. These events can cover various fields, such as natural disasters and social events. The goal of event detection is to extract specific event type information from massive text data to support tasks such as information extraction and information retrieval. Traditional event detection methods often use supervised learning techniques, which require each event type to be labeled. However, it is difficult and costly to obtain large-scale and high-quality labeled data, especially in the face of emerging unknown event types, which limits the applicability of event detection methods based on supervised learning.

[0003] In order to address the challenges faced by supervised learning-based event detection methods, the concept of zero-shot event detection was proposed, the purpose of which is to detect unknown event types. Zero-shot event detection methods usually utilize auxiliary information, among which contrastive learning and transfer learning are two common strategies. These strategies help the model effectively detect events by leveraging existing knowledge and data without relying on already labeled data. For example, Lyu et al. (Lyu Q, Zhang H, Sulem E, et al. Zero-shot Event Extraction via Transfer Learning: Challenges and Insights [C]. Proceedings of ACL [C]. 2021: 322-332.) achieved zero-shot event detection through transfer learning. They used textual implications and question-answering data to identify unknown event types. Zhang et al. (Zhang H, Yao W, Yu D. Efficient Zero-shot Event Extraction with Context-Definition Alignment [C] / / Findings of the Association for Computational Linguistics: EMNLP 2022. 2022: 7169-7179.) used two independent Transformer models to project event-related information and its definition in the context into the same embedding space, and then used contrastive learning to effectively detect previously unseen event types by reducing the distance in the embedding space. These methods demonstrate the potential of zero-shot event detection, which not only reduces the reliance on manual annotations by leveraging existing knowledge and techniques, but also improves the ability to detect unknown event types.

[0004] Although zero-shot event detection methods have achieved some success, they usually require additional auxiliary information. The most common type of auxiliary information is manually annotated features, such as trigger words and their positions. This fine-grained annotation work not only requires manual participation, but also the quality of the annotation is difficult to guarantee. Another type of auxiliary information is the relationship between event types, but using the relationship between events will significantly increase the complexity of the model, resulting in relatively low detection efficiency.

[0005] Therefore, technical personnel in this field are committed to developing a zero-sample event detection method based on identifying all classes based on a few classes, which can rely only on the annotation information of the event type, reduce the dependence on manual annotation, reduce the complexity of the model by selecting a prototype subset, and improve the efficiency and accuracy of zero-sample event detection. Summary of the invention

[0006] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is the limitation of zero-sample event detection.

[0007] To achieve the above object, the present invention provides a zero-sample event detection method for identifying all classes based on a minority class, the method comprising the following steps:

[0008] Step 1: Divide the data set into known event types and unknown event types;

[0009] Step 2: Encode the original text of the known event type;

[0010] Step 3: Obtain the prototypes of known event types through the prototype network to obtain a prototype set;

[0011] Step 4: Use the STIC algorithm to extract a prototype subset from the prototype set;

[0012] Step 5: Calculate the similarity between the data encoded from the original text and the prototype subset, and use each similarity value as a feature value to generate a feature data set;

[0013] Step 6: Encode the feature data set;

[0014] Step 7: Input the encoded feature dataset into the Transformer classifier for training and classification.

[0015] Furthermore, the step 2 further comprises:

[0016] The Sentence-Transformer pre-trained model is used as the encoder, which encodes the given input event sample x into an embedding vector Φ(x), where Φ represents the encoding process of the Sentence-Transformer pre-trained model.

[0017] Furthermore, the step 3 also includes:

[0018] Perform an average operation on the embedding vectors obtained for each event type to obtain the prototype representation of each event type;

[0019] The prototypes of all event types are stored in the prototype set P, where the prototype of the i-th event type is marked as p i .

[0020] Furthermore, the step 4 also includes:

[0021] Step 4.1, calculate the prototype set P = {p1, ..., p c}, and save the prototype pairs whose Euclidean distance is less than the threshold τ in D;

[0022] Step 4.2: If ω is empty, randomly select a prototype pair from D and add it to ω as a basic cluster, and remove the two prototypes from the prototype set P;

[0023] Step 4.3, calculate the similarity between the remaining prototypes in the prototype set P and all the prototypes in ω; if the similarity between a prototype in the prototype set P and all the prototypes in ω is less than τ, add this prototype to ω and remove this prototype from the prototype set P;

[0024] Step 4.4, repeating steps 4.1 to 4.3 until no new prototype is added to ω after traversing all prototypes in the prototype set P, at which point a cluster is constructed;

[0025] Step 4.5: Calculate the similarity between any two prototypes in the cluster, select the two prototypes with the largest similarity and add them to the prototype subset P s middle;

[0026] Step 4.6: If a prototype pair in D contains at least one prototype in ω, remove the prototype pair;

[0027] Step 4.7, set ω to an empty set, repeat steps 4.2 to 4.6, and continue to construct the next cluster until the prototype set P or D is empty.

[0028] Furthermore, if the prototypes in the prototype set are not selected into a cluster, then these prototypes are regarded as an independent cluster, that is, each prototype represents an independent cluster, and all these prototypes are added to the prototype subset.

[0029] Furthermore, in step 5, Euclidean distance is selected as the similarity calculation method.

[0030] Furthermore, the step 6 also includes:

[0031] Step 6.1, round the similarity values ​​in the feature data set to two decimal places;

[0032] Step 6.2, take each similarity value as a character and create a word list;

[0033] Step 6.3: Re-encode the feature dataset according to the vocabulary created in step 6.2.

[0034] Furthermore, the step 6.3 adopts a one-hot encoding method.

[0035] Furthermore, the step 7 also includes: training using a cross entropy loss function.

[0036] Furthermore, when detecting unknown event types, the similarity between the encoded data and the prototypes in the selected prototype subset is first calculated, and then each similarity value is regarded as a character, encoded using one-hot encoding, and finally input into the classifier for classification.

[0037] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0038] 1. The present invention effectively captures the similarities and differences between different known event types by using the representative prototype subset extracted by the prototype network and the STIC algorithm. This precise feature extraction and classification method enables the present invention to maintain highly accurate classification capabilities when facing unknown event types, thereby greatly reducing the false positive rate and missed detection rate. Moreover, since the present invention does not rely on a large amount of labeled sample data, it can effectively meet the needs of zero-sample event detection. This feature makes the present invention have important application prospects in emerging fields, rapidly changing market environments, or scenarios that need to quickly adapt to new event types.

[0039] 2. The present invention utilizes modern deep learning and natural language processing technologies and adopts the mature Transformer architecture as the core classifier. This model architecture has been verified and widely used in many large-scale applications. Its implementation and integration process is relatively standardized and mature, which helps to reduce the difficulty and risk of implementation.

[0040] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a M2M model framework diagram of a preferred embodiment of the present invention;

[0042] Figure 2 It is a schematic diagram of a similarity threshold iterative clustering algorithm of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0044] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0045] This embodiment provides a zero-sample event detection method based on identifying all classes based on a minority class, based on the proposed M2M (Minority-to-Majority model), such as Figure 1 As shown, it is a framework diagram of the M2M model, including a prototype generation module, a recoding module, and a classifier module.

[0046] First, the prototype generation module uses the prototype network to extract prototypes of known event types. This method is similar to the way humans identify unknown things by similarities between things, and learns and classifies by capturing the similarities between event types. Then, the STIC algorithm is used to further refine these prototypes and extract a representative subset of prototypes from the prototypes of all known event classes. This combination takes advantage of the prototype network and further optimizes the expression of the prototype through the clustering algorithm (STIC), thereby improving the model's adaptability and generalization ability for complex data sets.

[0047] Secondly, the re-encoding module generates a new feature dataset by calculating the similarity between the original text encoded data and the selected prototype subset. This similarity-based feature generation method helps to reduce the data dimension and effectively retains key information in the subsequent one-hot encoding process. Compared with the traditional direct text classification method, this feature generation method more effectively captures the semantic similarity between event types and improves the classification accuracy and robustness of the model when facing unknown event types.

[0048] Finally, the M2M model inputs the generated feature dataset into the classifier module for training and classification. The Transformer model is known for its excellent sequence modeling capabilities and self-attention mechanism, which can better handle long-distance dependencies and semantic information in text data. Compared with traditional classifiers, the Transformer classifier can more accurately understand and classify complex text data, thereby further improving the overall performance of the M2M model.

[0049] The M2M model has achieved good results in detecting both known event types and unknown event types.

[0050] Specifically, the detection method of the M2M model includes the following steps:

[0051] Step 1: The M2M model is mainly used for event detection tasks in sentence-level text. The dataset used is data extracted from social networks, such as the ACE2005 event dataset and the FewEvent event dataset. First, the dataset is divided into known event types and unknown event types.

[0052] Step 2: Encode the original text of the known event type and use the Sentence-Transformer pre-trained model in the M2M model for preliminary encoding.

[0053] Although the BERT pre-trained model is widely used in NLP tasks, it has certain limitations in acquiring semantic information, especially in calculating semantic similarity, which is not as good as the Sentence-Transformer pre-trained model. The Sentence-Transformer pre-trained model is known for its twin network structure and performs well in generating sentence embedding vectors with semantic meaning. Therefore, the Sentence-Transformer pre-trained model is chosen as the encoder. Specifically, for a given input event sample x, Sentence-Transformer encodes it into an embedding vector Φ(x), where Φ represents the encoding process of the Sentence-Transformer pre-trained model.

[0054] Step 3: Generate a prototype using the prototype network.

[0055] For each event type, we first obtain the encoded representations of all its instances. Then, we obtain the prototype representation of the event category by averaging these embedding vectors. The prototypes of all event types are stored in the prototype set P, where the prototype of the i-th event type is labeled p i .

[0056] Step 4: After obtaining c prototypes of all event types, a similarity threshold iterative clustering (STIC) algorithm is proposed to select a more representative prototype subset. Figure 2 As shown, it is a schematic diagram of the STIC algorithm, which includes the following steps:

[0057] Step 4.1, calculate P = {p1, ..., p c}, and save the prototype pairs whose Euclidean distance is less than the threshold τ in D;

[0058] Step 4.2: If ω is empty, randomly select a prototype pair from D, add it to ω as a basic cluster, and remove the two prototypes from the set P;

[0059] Step 4.3, calculate the similarity between the remaining prototypes in P and all the prototypes in ω; if the similarity between a prototype in P and all the prototypes in ω is less than τ, add this prototype to ω and remove this prototype from P;

[0060] Step 4.4, repeat steps 4.1 to 4.3 until after traversing all prototypes in P, if no new prototype is added to ω, then a basic cluster is constructed;

[0061] Step 4.5: Calculate the similarity between any two prototypes in this cluster, select the two prototypes with the largest similarity and add them to P s middle;

[0062] Step 4.6: If a prototype pair in D contains at least one prototype in ω, remove the prototype pair;

[0063] Step 4.7: Set ω to an empty set, repeat steps 4.2 to 4.6, and continue to construct the next cluster until P or D is empty.

[0064] Finally, some prototypes in P may not be selected into a cluster, so these prototypes are regarded as an independent cluster, that is, each prototype represents an independent cluster, and all these prototypes are added to P. s middle.

[0065] P s It is the prototype subset selected by the STIC algorithm, which contains m prototypes.

[0066] Step 5: Generate a new feature dataset using the embedding vector encoded by Sentence-Transformer and the m prototypes selected by the STIC algorithm.

[0067] Specifically, the similarity between the data encoded by Sentence-Transformer and the selected m prototypes is calculated, and the calculated similarity value is used as a character to create a feature dataset. For example, when an instance x is input into the Sentence-Transformer encoder, an embedding vector is generated, represented by Φ(x). Calculate Φ(x) and P s The similarity between the m prototypes in [Sim(p1,Φ(x)),…,Sim(p m ,Φ(x))], where Sim(p i ,Φ(x)) represents the text similarity calculation method. In the M2M model, Euclidean distance is selected as the similarity calculation method.

[0068] Step 6. After the feature dataset is generated, a new vocabulary is constructed for the feature dataset. In the feature dataset, each sentence in the original dataset is replaced by a set of corresponding similarity values. Since the similarity value is a decimal consisting of multiple decimal places, it has no practical meaning. To solve this problem, a strategy is adopted to round these similarity values ​​to two decimal places. Subsequently, each similarity value is taken as a character, thereby creating a smaller vocabulary. The efficiency of event detection is improved by creating a smaller dataset. Finally, the feature dataset is re-encoded according to the created vocabulary. Since the vocabulary is relatively small, One-Hot Encoding is adopted. This is an efficient encoding method suitable for datasets with limited vocabulary.

[0069] Step 7: Input the feature dataset into the Transformer classifier for training and classification.

[0070] Since its introduction, Transformer has been successfully applied to various fields of artificial intelligence and has shown excellent performance in various classification tasks. This wide range of applications is attributed to the ability of the Transformer model to capture relationships between sequences, including its ability to identify complex connections between individual words in a sentence. The obtained feature dataset is constructed based on the similarity between instances and a selected subset of prototypes. Each instance in the feature dataset consists of the similarity between the original encoded data and the selected subset of prototypes. In the feature dataset, there are certain relationships between these similarities, and the Transformer model is used to capture these relationships. Finally, the M2M model is trained using the classic cross entropy loss function.

[0071] The unknown event type data set used for unknown event type recognition is composed of event types that are not used in model training. When detecting unknown event types, the encoded data is firstly calculated with the prototypes in the selected prototype subset for similarity, and then each similarity value is regarded as a character, encoded using the same encoding method in the model, and finally input into the classifier for classification.

[0072] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A zero-shot event detection method based on identifying all classes from a few classes, characterized by: The method comprises the following steps: Step 1: Divide the data set into known event types and unknown event types; Step 2: Encode the original text of the known event type; Step 3: Obtain the prototypes of known event types through the prototype network to obtain a prototype set; Step 4: Use the STIC algorithm to extract a prototype subset from the prototype set; Step 5: Calculate the similarity between the data encoded from the original text and the prototype subset, and use each similarity value as a feature value to generate a feature data set; Step 6: Encode the feature data set; Step 7: Input the encoded feature dataset into the Transformer classifier for training and classification.

2. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 1, characterized in that: The step 2 also includes: The Sentence-Transformer pre-trained model is used as the encoder, which encodes the given input event sample x into an embedding vector Φ(x), where Φ represents the encoding process of the Sentence-Transformer pre-trained model.

3. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 2, characterized in that: The step 3 also includes: Perform an average operation on the embedding vectors obtained for each event type to obtain the prototype representation of each event type; The prototypes of all event types are stored in the prototype set P, where the prototype of the i-th event type is marked as p i .

4. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 3, characterized in that: The step 4 also includes: Step 4.1, calculate the prototype set P = {p1, ..., p c }, and save the prototype pairs whose Euclidean distance is less than the threshold τ in D; Step 4.2: If ω is empty, randomly select a prototype pair from D and add it to ω as a basic cluster, and remove the two prototypes from the prototype set P; Step 4.3, calculate the similarity between the remaining prototypes in the prototype set P and all the prototypes in ω; if the similarity between a prototype in the prototype set P and all the prototypes in ω is less than τ, add this prototype to ω and remove this prototype from the prototype set P; Step 4.4, repeating steps 4.1 to 4.3 until no new prototype is added to ω after traversing all prototypes in the prototype set P, at which point a cluster is constructed; Step 4.5: Calculate the similarity between any two prototypes in the cluster, select the two prototypes with the largest similarity and add them to the prototype subset P s middle; Step 4.6: If a prototype pair in D contains at least one prototype in ω, remove the prototype pair; Step 4.7, set ω to an empty set, repeat steps 4.2 to 4.6, and continue to construct the next cluster until the prototype set P or D is empty.

5. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 4, characterized in that: If the prototypes in the prototype set are not selected into a cluster, then these prototypes are regarded as an independent cluster, that is, each prototype represents an independent cluster, and all these prototypes are added to the prototype subset.

6. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 1, characterized in that: In step 5, Euclidean distance is selected as the similarity calculation method.

7. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 1, characterized in that: The step 6 also includes: Step 6.1, round the similarity values ​​in the feature data set to two decimal places; Step 6.2, take each similarity value as a character and create a word list; Step 6.3: Re-encode the feature dataset according to the vocabulary created in step 6.

2.

8. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 7, characterized in that: The step 6.3 adopts one-hot encoding.

9. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 1, characterized in that: The step 7 also includes: using a cross entropy loss function for training.

10. The zero-sample event detection method based on identifying all classes from a minority class as claimed in claim 1, characterized in that: When detecting unknown event types, the similarity between the encoded data and the prototypes in the selected prototype subset is first calculated, and then each similarity value is regarded as a character, encoded using one-hot encoding, and finally input into the classifier for classification.