A knowledge graph-oriented overlapping event extraction system and method

Abstract

The present invention discloses a knowledge graph-oriented overlapping event extraction system and method. The system includes a BERT encoder, an event type recognition decoder, an event entity recognition decoder, a trigger prediction decoder, and a parameter prediction decoder; the method includes feature extraction, event type recognition, event entity recognition, trigger prediction, and parameter prediction. The present invention adopts a hierarchical step-by-step decoding method to gradually reduce the judgment scope of event extraction; at the same time, the decoding results of the previous level can be used at each level to further improve the accuracy of event extraction. The present invention can gradually identify and extract features of different levels of events in the order of type detection, entity detection, trigger recognition, and parameter recognition, thereby more accurately identifying and judging events and avoiding conflicts and confusions of overlapping events.

Description

Technical Field

[0001] The present invention relates to the field of information extraction technology, and in particular to a knowledge graph-oriented overlapping event extraction method and system. Background Art

[0002] With the widespread adoption and development of the internet, the amount of information on the web is growing exponentially. How to quickly and accurately extract the required information from this vast amount of natural language text has become a pressing task. Information extraction (IE) technology can extract specific types of factual information, such as entities, relationships, and events, from large amounts of unstructured text in the form of structured data. Event extraction is one of the most challenging tasks, aiming to extract basic event information, such as the type of event, participants, time and location of the event, from unstructured natural language text and present it in a structured form.

[0003] In the process of event extraction, we often encounter situations where multiple events occur simultaneously or partially overlap in the text within the same time or continuous time period. Currently, there is no effective method for extracting overlapping events, resulting in low accuracy in event extraction for overlapping parts. Summary of the Invention

[0004] In view of this, an embodiment of the present invention provides a knowledge graph-oriented overlapping event extraction system and method.

[0005] A first aspect of the present invention provides a knowledge graph-oriented overlapping event extraction system, comprising a BERT encoder, an event type recognition decoder, an event entity recognition decoder, a trigger prediction decoder, and a parameter prediction decoder;

[0006] The BERT encoder is used to extract features from a sentence with multiple tags to obtain token representations, trigger tags, and relative position tags;

[0007] The event type identification decoder is used to identify the event type of the token representation to obtain event information represented by the token; the event information includes a predicted probability distribution of each event type and an event type embedding vector;

[0008] The event entity recognition decoder is used to perform entity task recognition on the token representation to obtain entity information represented by the token; the entity information includes entity position probability distribution and entity type embedding vector;

[0009] The trigger prediction decoder is used to fuse the token representation, the entity probability position distribution and the event type embedding vector to obtain a trigger position probability distribution and a feature representation; the trigger position probability distribution is used to perform trigger prediction;

[0010] The parameter prediction decoder is used to perform parameter decoding based on the feature representation, the trigger mark, the relative position mark, the event type embedding vector and the entity type embedding vector to obtain event type constraints and parameter position probability distribution; the parameter position probability distribution is used for parameter prediction.

[0011] Furthermore, the BERT encoder is established based on the Bert-base-chinese architecture; the sentence with multiple tags is input into the BERT encoder, and the BERT encoder obtains the semantic representation of the hidden state corresponding to each tag in the sentence; the semantic representation of the hidden state is used as a token representation for event type recognition, entity recognition and trigger prediction; the BERT encoder also obtains the trigger tag of the sentence through random masking, and obtains the relative position tag of the sentence through relative position encoding.

[0012] Furthermore, the event type identification decoder includes a first Embedding layer and an AdaptiveAdditionPredictor layer;

[0013] The event type identification decoder performs event type identification decoding by the following steps:

[0014] The first Embedding layer performs vector mapping according to the preset number of event categories to obtain an event type embedding vector; the AdaptiveAdditionPredictor layer receives the token representation and the event type embedding vector generated by the first Embedding layer, performs adaptive addition prediction, and obtains a predicted probability distribution for each event type; the event type embedding vector and the predicted probability distribution of each event type are output from the event type recognition decoder.

[0015] Furthermore, the event entity recognition decoder includes a BiGRU layer, a first linear layer, a tanh layer, a second linear layer, a softmax layer, and a third linear layer; the BiGRU layer, the first linear layer, the tanh layer, the second linear layer, and the softmax layer are sequentially connected; the tanh layer is also connected to the third linear layer;

[0016] The event entity recognition decoder performs entity recognition decoding through the following steps:

[0017] After inputting the token representation into the BiGRU layer for bidirectional GRU decoding, a forward hidden state and a backward hidden state of the token representation are obtained; the forward hidden state and the backward hidden state are concatenated to form an entity hidden state;

[0018] Using a preset weight matrix and bias vector to perform a tanh activation function operation and a softmax normalization operation on the entity hidden state to obtain an entity position probability distribution and an entity type embedding vector; the entity position probability distribution includes an entity start position probability distribution and an entity end position probability distribution;

[0019] The entity position probability distribution is output in the softmax layer, and the entity type embedding vector is output in the third linear layer; the entity position probability distribution and the entity type embedding vector are output from the event entity recognition decoder as entity information.

[0020] Furthermore, the trigger prediction decoder includes a Weighting layer, a Self_Attention layer, a first ConditionIntergrator layer, a first MultiHeadedAttention layer, a LayerNorm layer, a first gelu layer, a dropout layer, and a fourth Linear layer;

[0021] The trigger prediction decoder performs trigger prediction decoding by the following steps:

[0022] After inputting the entity probability distribution into the Weighting layer for weighting, the weighted entity probability distribution is fused with the token representation in the Self_Attention layer to obtain a first intermediate representation;

[0023] fusing the first intermediate representation with the event type embedding vector in the first ConditionIntergrator layer to obtain a feature representation;

[0024] After inputting the feature representation into the first MultiHeadedAttention layer for attention-based feature extraction, the feature representation is combined with the original feature representation and sequentially passed through the LayerNorm layer, the first gelu layer, the dropout layer, and the fourth Linear layer to obtain the trigger position probability distribution; the trigger position probability distribution includes the trigger start position probability distribution and the trigger end position probability distribution;

[0025] The trigger position probability distribution and the feature representation are output from the trigger prediction decoder.

[0026] Furthermore, the parameter prediction decoder includes a second MultiHeadedAttention layer, a bmm layer, a second Embedding layer, a second ConditionIntergrator layer, a third MultiHeadedAttention layer, a LayerNorm layer, a cat layer, a second gelu layer, a fifth Linear layer, a sixth Linear layer and a Constraint layer;

[0027] The parameter prediction decoder performs parameter prediction decoding by the following steps:

[0028] Inputting the feature representation and the entity type embedding vector into the second MultiHeadedAttention layer to perform attention-based feature extraction to obtain a second intermediate representation;

[0029] Input the feature representation and the trigger flag into the bmm layer to perform a matrix multiplication operation to obtain a representation matrix;

[0030] Input the relative position mark into the second Embedding layer to perform vector mapping to obtain an intermediate vector;

[0031] Input the event type embedding vector into the fifth linear layer for linear transformation to obtain event type constraints;

[0032] The second intermediate representation and the representation matrix are input into the second ConditionIntergrator layer for conditional normalization, and then sequentially passed through the third MultiHeadedAttention layer and the LayerNorm layer, and concatenated with the intermediate vector in the cat layer to obtain a representation matrix;

[0033] After the representation matrix passes through the second gelu layer and the sixth Linear layer in sequence, the parameter range is constrained according to the event type constraint in the Constraint layer to obtain the parameter position probability distribution; the parameter position probability distribution includes the parameter starting position probability distribution and the parameter ending position probability distribution;

[0034] The parameter position probability distribution and the event type constraint are output from the parameter prediction decoder.

[0035] The second aspect of the present invention discloses a method for extracting overlapping events based on a knowledge graph, comprising the following steps:

[0036] Extract features from sentences with multiple tokens to obtain token representations, trigger tags, and relative position tags;

[0037] Performing event type recognition on the token representation to obtain event information represented by the token; the event information includes a predicted probability distribution of each event type and an event type embedding vector;

[0038] Performing entity task recognition on the token representation to obtain entity information represented by the token; the entity information includes entity position probability distribution and entity type embedding vector;

[0039] Fusion transforms the token representation, the entity probability position distribution, and the event type embedding vector to obtain a trigger position probability distribution and feature representation;

[0040] Perform parameter decoding according to the feature representation, the trigger mark, the relative position mark, the event type embedding vector, and the entity type embedding vector to obtain event type constraints and parameter position probability distribution;

[0041] Perform trigger prediction based on the trigger position probability distribution;

[0042] Parameter prediction is performed based on the parameter position probability distribution.

[0043] Furthermore, the trigger prediction specifically includes the following steps:

[0044] Set preset trigger probability threshold distribution;

[0045] Compare the area range of the trigger position probability distribution with the trigger probability threshold distribution; and use the trigger position probability distribution whose distribution range covers the trigger probability threshold distribution as the trigger predicted by the corresponding tag.

[0046] Furthermore, the parameter prediction specifically includes the following steps:

[0047] defining an indicator function, the indicator function being used to determine the event type to which the tag belongs;

[0048] Substitute the parameter position probability distribution into the indicator function to obtain parameter information corresponding to the marker prediction of the parameter position probability distribution.

[0049] Furthermore, after completing the trigger prediction and the parameter prediction, the following steps are further included:

[0050] Based on the predicted triggers and parameters, event extraction is performed on the sentence with multiple tags.

[0051] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0052] The embodiments of the present invention have the following beneficial effects: A knowledge graph-based overlapping event extraction method and system employ a layered, step-by-step decoding approach to gradually narrow the scope of event extraction. Simultaneously, each layer leverages the decoding results of the previous layer, further improving the accuracy of event extraction. The present invention progressively identifies and extracts features at different levels of an event, following the sequence of type detection, entity detection, trigger identification, and parameter identification. This allows for more accurate event identification and judgment, avoiding conflicts and confusion among overlapping events.

[0053] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0055] Figure 1 This is a basic implementation flow chart of a knowledge graph-oriented overlapping event extraction system and method of the present invention;

[0056] Figure 2 Schematic diagram of an event type recognition decoder in a knowledge graph-oriented overlapping event extraction system according to the present invention;

[0057] Figure 3 Schematic diagram of an event entity recognition decoder in a knowledge graph-oriented overlapping event extraction system according to the present invention;

[0058] Figure 4 Schematic diagram of a trigger prediction decoder in a knowledge graph-oriented overlapping event extraction system according to the present invention;

[0059] Figure 5 Schematic diagram of a parameter prediction decoder in a knowledge graph-oriented overlapping event extraction system according to the present invention;

[0060] Figure 6This is a flowchart of the basic steps of a knowledge graph-oriented overlapping event extraction method of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0062] The purpose of the present invention is to address the problem of event extraction of overlapping events, and to introduce entity information obtained by entity recognition to assist in the identification of triggers and parameters. Entity recognition technology can effectively identify named entities in texts, such as names of people, organizations, and places, which are usually the main information required for event extraction tasks. On the one hand, the location information of the entity can effectively assist in the positioning of the trigger. On the other hand, entity information can provide valuable contextual information and location information for parameter prediction, helping us better understand the meaning and purpose of the text. Therefore, entity recognition can help us locate the extraction triggers and parameters more accurately, and improve the accuracy of information extraction.

[0063] like Figure 1 As shown, an embodiment of the present invention discloses an overlapping event extraction system for knowledge graphs, including a BERT encoder, an event type recognition decoder, an event entity recognition decoder, a trigger prediction decoder, and a parameter prediction decoder.

[0064] (1) BERT encoder: The BERT encoder is used to extract features from sentences with multiple tags to obtain token representations, trigger tags, and relative position tags. In this embodiment, the BERT (Bidirectional Encoder Representations from Transformers) model is a bidirectional encoder with a Transformer structure, which is built based on the Bert-base-chinese architecture and pre-trained based on large-scale Chinese text data such as Chinese Wikipedia. It can have 12 encoder layers (transformer blocks) and 110 million parameters, which is suitable for Chinese text processing tasks. In this embodiment, a sentence with multiple tags x = {w1, w2, ..., w N} Input BERT encoder, BERT encoder obtains the semantic representation H = {h1,h2,…,h N}; The semantic representation of the hidden state is used as a token representation (text_rep) for event type recognition, entity recognition, and trigger prediction.

[0065] In this embodiment, the BERT encoder performs a random mask operation by randomly masking or replacing one or more tokens in a sentence. The BERT encoder predicts the masked or replaced parts of the sentence based on contextual understanding, and obtains the trigger token (trigger_mask) of the sentence based on the prediction results.

[0066] The BERT encoder in this embodiment performs relative position encoding (RPR), encoding the relative positions of each token in a sentence, learning the positional relationships between tokens, and forming a relative position marker (Relative_pos) for the sentence. The token representation (text_rep), trigger mask (trigger_mask), and relative position marker (Relative_pos) are used as the output of the BERT encoder.

[0067] Event type identification decoder: such as Figure 2 As shown, the event type recognition decoder is used to identify the event type of the token representation (text_rep) to obtain the event information of the token representation (text_rep); the event information includes the predicted probability distribution of each event type and the event type embedding vector.

[0068] In this embodiment, the event type recognition decoder is a TypeCls decoder, which includes a first Embedding layer and an AdaptiveAdditionPredictor layer. The first Embedding layer performs vector mapping according to the preset number of event categories to obtain an event type embedding vector (type_emb), and each event type embedding vector represents a different event type. Secondly, the AdaptiveAdditionPredictor layer (adaptive addition predictor) is used to combine the text representation and category information, calculate the correlation between them, and obtain the probability distribution of each category. The AdaptiveAdditionPredictor layer uses an attention mechanism to focus on the most relevant contextual information in the text, and measures the correlation between the event type and the token representation by defining a similarity function to obtain the predicted probability distribution of each event type.

[0069] δ(c,h i )=v T tanh(W[c;h i ;|ch i |;c⊙h i ])

[0070]

[0071] In the formula, W∈R4d×4d and v∈R 4d×1 is a learnable parameter, [·;·] represents a connection, ⊙ is element-by-element generation, i.e., the product at the element level, c represents the event type, h i represents the i-th hidden state. The predicted probability distribution of each event type is used express.

[0072] The predicted probability distribution (p_type) and event type embedding vector (type_emb) of each event type are taken as the output of the event type recognition decoder.

[0073] Event Entity Recognition Decoder: Figure 3 As shown, in this embodiment, the event entity recognition decoder is used to perform entity task recognition on the token representation (text_rep) to obtain the entity information represented by the token; the entity information includes the entity position probability distribution (p_s, p_e) and the entity type embedding vector (entity_type_emb);

[0074] In this embodiment, the event entity recognition decoder is a bidirectional GRU decoder (BiGRUDecoder); it includes a BiGRU layer, a first linear layer, a tanh layer, a second linear layer, a softmax layer, and a third linear layer; the BiGRU layer, the first linear layer, the tanh layer, the second linear layer, and the softmax layer are connected in sequence; the tanh layer is also connected to the third linear layer;

[0075] In this embodiment, the event entity recognition decoder inputs the token representation into the BiGRU layer for bidirectional GRU decoding to obtain the forward hidden state of the token representation. and the backward hidden state

[0076]

[0077]

[0078] Concatenate the forward hidden state and the backward hidden state to obtain the entity hidden state;

[0079]

[0080] Use the preset weight matrix (W s 、W e ) and the bias vector (b s 、b e) Perform tanh activation function operation and softmax normalization operation on the entity hidden state to obtain the entity position probability distribution and entity type embedding vector; the entity position probability distribution includes the entity starting position probability distribution (p_s) and the entity ending position probability distribution (p_e);

[0081] Through the weight matrix (W h ) and the bias vector (b h )Hide state for entities Perform the operation and get the entity type embedding vector (entity_type_emb) after processing with the tanh function

[0082] p_s=softmax(tanh(W s H+b s ))

[0083] p_e=softmax(tanh(W e H+b e ))

[0084] entity_type_emb=tanh(W h H+b h )

[0085] The entity position probability distribution is output in the softmax layer, and the entity type embedding vector is output in the third linear layer; the entity position probability distribution (p_s, p_e) and the entity type embedding vector (entity_type_emb) are output as entity information from the event entity recognition decoder.

[0086] Trigger prediction decoder: Figure 4 As shown, the trigger prediction decoder of this embodiment is used to fuse the token representation (text_rep), the entity probability position distribution (p_s, p_e) and the event type embedding vector (type_rep) to obtain the trigger position probability distribution (p_s1, p_e1) and the feature representation (text_rep_type); the trigger position probability distribution is used to perform trigger prediction;

[0087] In this embodiment, the trigger prediction decoder is a TriggerRec decoder, which includes a Weighting layer, a Self_Attention layer, a first ConditionIntergrator layer, a first MultiHeadedAttention layer, a LayerNorm layer, a first gelu layer, a dropout layer and a fourth Linear layer.

[0088] In this embodiment, after the entity probability distribution is input into the Weighting layer for weighting, the weighted entity probability distribution is fused with the token representation in the Self_Attention layer to obtain the first intermediate representation; secondly, the first intermediate representation is fused with the event type embedding vector in the first ConditionIntergrator layer to obtain the feature representation (text_rep_type).

[0089] In this embodiment, conditional layer normalization (CLN) is used to implement the conditional fusion function, by incorporating the type of embedding c into the token representation h i , obtain feature representation

[0090]

[0091] where γ c =W γ c+b γ , β c =W β c+b β , c represents the type embedding vector, W γ 、W β is the weight matrix, b γ 、b β is the bias vector.

[0092] After obtaining the feature representation, the trigger prediction decoder inputs the feature representation into the first MultiHeadedAttention layer for attention-based feature extraction. It then combines the feature representation with the original feature representation and passes through the LayerNorm layer (layer normalization), the first gelu layer (activation function), the dropout layer, and the fourth Linear layer to obtain the trigger position probability distribution (p_s1, p_e1). The trigger position probability distribution includes the trigger start position probability distribution p_s1 and the trigger end position probability distribution p_e1.

[0093] The trigger position probability distribution (p_s1, p_e1) and feature representation (text_rep_type) are output as the trigger prediction decoder.

[0094] Parameter prediction decoder: such as Figure 5As shown, the parameter prediction decoder of this embodiment is used to perform parameter decoding based on the feature representation (text_rep_type), trigger mark (trigger_mask), relative position mark (relative_pos), event type embedding vector (type_emb) and entity type embedding vector (entity_type_emb), and obtain the event type constraint (type_soft_constrain) and parameter position probability distribution (p_s2, p_e2); the parameter position probability distribution is used for parameter prediction.

[0095] In this embodiment, the parameter prediction decoder is an ArgsRec decoder, which includes a second MultiHeadedAttention layer, a bmm layer, a second Embedding layer, a second ConditionIntergrator layer, a third MultiHeadedAttention layer, a LayerNorm layer, a cat layer, a second gelu layer, a fifth Linear layer, a sixth Linear layer and a Constraint layer.

[0096] Specifically, first, the feature representation and entity type embedding vector are input into the second MultiHeadedAttention layer for attention-based feature extraction to obtain the second intermediate representation; the feature representation and trigger mark are input into the bmm layer for matrix multiplication to obtain the representation matrix (trigger_emb); the relative position mark is input into the second Embedding layer for vector mapping to obtain the intermediate vector (rep_emb); the event type embedding vector is input into the fifth Linear layer for linear transformation to obtain the event type constraint (type_soft_constrain).

[0097] Secondly, the second intermediate representation and representation matrix are input into the second ConditionIntergrator layer for conditional normalization, and then pass through the third MultiHeadedAttention layer and LayerNorm layer in sequence, and are concatenated with the intermediate vector in the cat layer to obtain the representation matrix (inp);

[0098] After the representation matrix passes through the second gelu layer and the sixth Linear layer, the parameter range is constrained according to the event type constraint at the Constraint layer to obtain the parameter position probability distribution (p_s2, p_e2); the parameter position probability distribution includes the parameter starting position probability distribution p_s2 and the parameter ending position probability distribution p_e2;

[0099] The parameter position probability distribution and event type constraints are taken as the output of the parameter prediction decoder.

[0100] The embodiment of the present invention discloses a knowledge graph-oriented overlapping event extraction method, such as Figure 6 As shown, the following steps are included:

[0101] S1. Extract features from sentences with multiple tokens to obtain token representations, trigger tags, and relative position tags.

[0102] S2. Identify the event type of the token representation to obtain the event information represented by the token; the event information includes the predicted probability distribution of each event type and the event type embedding vector;

[0103] S3. Perform entity task recognition on the token representation to obtain entity information represented by the token; the entity information includes the probability distribution of entity position and the entity type embedding vector;

[0104] S4. Fusion transforms the token representation, entity probability position distribution, and event type embedding vector to obtain the trigger position probability distribution and feature representation;

[0105] S5. Decode parameters based on feature representation, trigger tag, relative position tag, event type embedding vector, and entity type embedding vector to obtain event type constraints and parameter position probability distribution;

[0106] S6. Perform trigger prediction based on the trigger position probability distribution;

[0107] S7. Predict parameters based on the probability distribution of parameter positions.

[0108] The implementation process of steps S1-S5 in this method embodiment is the same as the BERT encoder, event type recognition decoder, event entity recognition decoder, trigger prediction decoder and parameter prediction decoder in the above-mentioned system embodiment, and the beneficial effects produced are also the same.

[0109] In step S6, trigger prediction specifically includes the following steps:

[0110] S6-1. Set the preset trigger probability threshold distribution;

[0111] S6-2. Compare the regional range of the trigger position probability distribution and the trigger probability threshold distribution; use the trigger position probability distribution whose distribution range covers the trigger probability threshold distribution as the trigger predicted by the corresponding tag.

[0112] In this embodiment, step S6 completes the identification of the trigger by setting a threshold, and by comparing the area range of the trigger position probability distribution and the trigger probability threshold distribution, the trigger position probability distribution that completely covers the trigger probability threshold distribution is selected as the trigger for the corresponding tag prediction.

[0113] In step S7, parameter prediction specifically includes the following steps:

[0114] S7-1 defines an indicator function, an indicator function for determining the event type to which the tag belongs; the indicator function defined in this embodiment is as follows:

[0115]

[0116] Where r represents the tag, c represents the event type, and w r 、b r Represents the weight matrix and bias vector, and the superscript T represents the transpose.

[0117] S7-2. Substitute the parameter position probability distribution into the indicator function to obtain the parameter information corresponding to the marker prediction. By deducing the indicator function, the parameter position probability distribution can be used to obtain the parameter information corresponding to the marker prediction, establishing an association between the marker and the event type.

[0118] After completing trigger prediction and parameter prediction, this embodiment further includes the following steps:

[0119] S8. Event extraction for sentences with multiple tokens based on predicted triggers and parameters.

[0120] This method uses multi-layer feature recognition to predict triggers and parameters for overlapping events within a sentence. It gradually identifies and extracts features at different levels of an event, enabling more accurate event recognition and judgment, while avoiding conflicts and confusion among overlapping events. In particular, the method adds entity location and type information to assist in trigger and parameter identification during event extraction, improving the accuracy of model predictions.

[0121] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 6 The method shown.

[0122] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0123] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0124] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0125] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0126] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A knowledge graph-oriented overlapping event extraction system, characterized by: Includes BERT encoder, event type recognition decoder, event entity recognition decoder, trigger prediction decoder, and parameter prediction decoder; The BERT encoder is used to extract features from a sentence with multiple tags to obtain token representations, trigger tags, and relative position tags; The event type identification decoder is used to identify the event type of the token representation to obtain event information represented by the token; the event information includes a predicted probability distribution of each event type and an event type embedding vector; The event entity recognition decoder is used to perform entity task recognition on the token representation to obtain entity information represented by the token; the entity information includes entity position probability distribution and entity type embedding vector; The trigger prediction decoder is used to fuse the token representation, the entity position probability distribution and the event type embedding vector to obtain a trigger position probability distribution and a feature representation; the trigger position probability distribution is used to perform trigger prediction; The parameter prediction decoder is used to perform parameter decoding based on the feature representation, the trigger mark, the relative position mark, the event type embedding vector and the entity type embedding vector to obtain event type constraints and parameter position probability distribution; the parameter position probability distribution is used for parameter prediction.

2. The overlapping event extraction system for knowledge graph according to claim 1 is characterized in that: The BERT encoder is built based on the Bert-base-chinese architecture; the sentence with multiple tags is input into the BERT encoder, and the BERT encoder obtains the semantic representation of the hidden state corresponding to each tag in the sentence; the semantic representation of the hidden state is used as a token representation for event type recognition, entity recognition and trigger prediction; the BERT encoder also obtains the trigger tag of the sentence through random masking and obtains the relative position tag of the sentence through relative position encoding.

3. The overlapping event extraction system for knowledge graph according to claim 1 is characterized in that: The event type identification decoder includes a first Embedding layer and an AdaptiveAdditionPredictor layer; The event type identification decoder performs event type identification decoding by the following steps: The first Embedding layer performs vector mapping according to the preset number of event categories to obtain an event type embedding vector; the AdaptiveAdditionPredictor layer receives the token representation and the event type embedding vector generated by the first Embedding layer, performs adaptive addition prediction, and obtains a predicted probability distribution for each event type; the event type embedding vector and the predicted probability distribution of each event type are output from the event type recognition decoder.

4. The knowledge graph-oriented overlapping event extraction system according to claim 1, characterized in that: The event entity recognition decoder includes a BiGRU layer, a first linear layer, a tanh layer, a second linear layer, a softmax layer, and a third linear layer; the BiGRU layer, the first linear layer, the tanh layer, the second linear layer, and the softmax layer are sequentially connected; the tanh layer is also connected to the third linear layer; The event entity recognition decoder performs entity recognition decoding through the following steps: After inputting the token representation into the BiGRU layer for bidirectional GRU decoding, a forward hidden state and a backward hidden state of the token representation are obtained; the forward hidden state and the backward hidden state are concatenated to form an entity hidden state; Using a preset weight matrix and bias vector to perform a tanh activation function operation and a softmax normalization operation on the entity hidden state to obtain an entity position probability distribution and an entity type embedding vector; the entity position probability distribution includes an entity start position probability distribution and an entity end position probability distribution; The entity position probability distribution is output in the softmax layer, and the entity type embedding vector is output in the third linear layer; the entity position probability distribution and the entity type embedding vector are output from the event entity recognition decoder as entity information.

5. The overlapping event extraction system for knowledge graph according to claim 1 is characterized in that: The trigger prediction decoder includes a Weighting layer, a Self_Attention layer, a first ConditionIntergrator layer, a first MultiHeadedAttention layer, a LayerNorm layer, a first gelu layer, a dropout layer and a fourth Linear layer; The trigger prediction decoder performs trigger prediction decoding by the following steps: After inputting the entity position probability distribution into the Weighting layer for weighting, the weighted entity position probability distribution is fused with the token representation in the Self_Attention layer to obtain a first intermediate representation; fusing the first intermediate representation with the event type embedding vector in the first ConditionIntergrator layer to obtain a feature representation; After inputting the feature representation into the first MultiHeadedAttention layer for attention-based feature extraction, the feature representation is combined with the original feature representation and sequentially passed through the LayerNorm layer, the first gelu layer, the dropout layer, and the fourth Linear layer to obtain the trigger position probability distribution; the trigger position probability distribution includes the trigger start position probability distribution and the trigger end position probability distribution; The trigger position probability distribution and the feature representation are output from the trigger prediction decoder.

6. The knowledge graph-oriented overlapping event extraction system according to claim 1, characterized in that: The parameter prediction decoder includes a second MultiHeadedAttention layer, a bmm layer, a second Embedding layer, a second ConditionIntergrator layer, a third MultiHeadedAttention layer, a LayerNorm layer, a cat layer, a second gelu layer, a fifth Linear layer, a sixth Linear layer and a Constraint layer; The parameter prediction decoder performs parameter prediction decoding by the following steps: Inputting the feature representation and the entity type embedding vector into the second MultiHeadedAttention layer to perform attention-based feature extraction to obtain a second intermediate representation; Input the feature representation and the trigger flag into the bmm layer to perform a matrix multiplication operation to obtain a representation matrix; Input the relative position mark into the second Embedding layer to perform vector mapping to obtain an intermediate vector; Input the event type embedding vector into the fifth linear layer for linear transformation to obtain event type constraints; The second intermediate representation and the representation matrix are input into the second ConditionIntergrator layer for conditional normalization, and then sequentially passed through the third MultiHeadedAttention layer and the LayerNorm layer, and concatenated with the intermediate vector in the cat layer to obtain a representation matrix; After the representation matrix passes through the second gelu layer and the sixth Linear layer in sequence, the parameter range is constrained according to the event type constraint in the Constraint layer to obtain the parameter position probability distribution; the parameter position probability distribution includes the parameter starting position probability distribution and the parameter ending position probability distribution; The parameter position probability distribution and the event type constraint are output from the parameter prediction decoder.

7. A knowledge graph-based overlapping event extraction method, characterized in that: The following steps are involved: Extract features from sentences with multiple tokens to obtain token representations, trigger tags, and relative position tags; Performing event type identification on the token representation to obtain event information represented by the token; The event information includes a predicted probability distribution of each event type and an event type embedding vector; Performing entity task recognition on the token representation to obtain entity information represented by the token; the entity information includes entity position probability distribution and entity type embedding vector; fusing the token representation, the entity position probability distribution, and the event type embedding vector to obtain a trigger position probability distribution and feature representation; Perform parameter decoding according to the feature representation, the trigger mark, the relative position mark, the event type embedding vector, and the entity type embedding vector to obtain event type constraints and parameter position probability distribution; Perform trigger prediction based on the trigger position probability distribution; Parameter prediction is performed based on the parameter position probability distribution.

8. The method for extracting overlapping events based on a knowledge graph according to claim 7, characterized in that: The trigger prediction specifically includes the following steps: Set preset trigger probability threshold distribution; Compare the area range of the trigger position probability distribution with the trigger probability threshold distribution; and use the trigger position probability distribution whose distribution range covers the trigger probability threshold distribution as the trigger predicted by the corresponding tag.

9. The method for extracting overlapping events based on a knowledge graph according to claim 7, wherein: The parameter prediction specifically includes the following steps: defining an indicator function, the indicator function being used to determine the event type to which the tag belongs; Substitute the parameter position probability distribution into the indicator function to obtain parameter information corresponding to the marker prediction of the parameter position probability distribution.

10. The method for extracting overlapping events based on a knowledge graph according to claim 7, characterized in that: After completing the trigger prediction and the parameter prediction, the following steps are further included: Based on the predicted triggers and parameters, event extraction is performed on the sentence with multiple tags.

Images