Method for generating discourse-level event timeline
Through the discourse-level event timing relationship extraction method based on heterogeneous graphs and multi-grained context encoding, the discourse parameter dispersion and timing relationship consistency problems in discourse-level event extraction and timing relationship extraction are solved. Improves the generation accuracy of event timelines.
Patent Information
- Application Number
- CN202410548830.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-06
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-05-06
AI Technical Summary
In the discourse level event extraction and timing relationship extraction, there are problems such as discourse parameter dispersion, multi-event argument parameter assembly, non-adjacent sentence event timing relationship recognition and global consistency of timing relationships, resulting in discourse level The accuracy and recall of event extraction and timing relationship extraction are low.
Using discourse-level event extraction method based on heterogeneous graphs and discourse-level event timing relationship extraction method based on multi-grained context encoding, the argument parameters are explicitly captured by constructing a heterogeneous graph model, and graph conversion network and multi-grained context encoding are used. The machine processes event elements and timing information separately and generates event timelines.
It effectively solves the problems of argument parameter dispersion and multi-event assembly, improves the accuracy and global consistency of discourse-level event timing relationship extraction, and improves the accuracy of discourse-level event timeline generation.
Smart Images

Figure CN118964627A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer application, and in particular relates to a method for generating a discourse-level event timeline. Background Art
[0002] In today's era, the rapid development of information technology has greatly improved the level of social productivity. Technologies represented by artificial intelligence, cloud computing, and the Internet of Things have ushered in a new round of industrial revolution. As the most representative technology in the new round of industrial revolution, artificial intelligence enables machines to have intelligent behavior and thinking, greatly improving the production efficiency of human society. Natural language processing is an important branch of artificial intelligence. Its purpose is to enable machines to understand and generate human language. Natural language processing involves many fields and has broad application prospects in many industries, such as social media, financial analysis, and e-commerce.
[0003] With the trend of digital transformation in many industries, a large amount of text data has emerged, such as news, electronic medical records, scientific literature, etc. Most of these data exist in unstructured form, and manual management and utilization of these data are extremely cumbersome and labor-intensive. Using information extraction technology to extract effective information from massive text data and convert unstructured data into structured data is helpful for data storage, query, and analysis. For example, information extraction technology can extract entities, relations, events, emotions, topics, keywords, summaries, etc. from natural language text. Events are language units that describe real-world situations. Compared with entities, information extraction based on events as basic units can more effectively reflect text content and logical relationships. Information extraction based on events as basic units includes event extraction and event relationship extraction. Researchers usually combine event extraction with event temporal relationship extraction to reveal the connection and influence between events and understand the logic and structure of the text, that is, to generate event timelines. Event timeline generation aims to extract event argument parameters and event pair temporal relationships from text, and display event details in the form of a timeline. This direction has become a research hotspot in the fields of finance, law, and medicine. In the past, most event extraction and event temporal relationship extraction technologies were based on sentence-level text. In comparison, discourse-level technologies are more in line with the actual needs of various fields. For example, for discourses such as news reports and historical research, the text content and logical structure are sorted out by generating discourse-level event timelines; for application scenarios such as public opinion monitoring and financial analysis, the development of events is tracked and future trends of events are predicted by extracting discourse-level event timelines. In addition, discourse-level event timeline generation technology also plays a vital role in downstream research tasks. For example, the event timeline is combined with the question-answering system to generate a text summary; the event elements and event relationship information in the event timeline are embedded in the graph to construct a temporal knowledge graph.
[0004] Event extraction and event temporal relation extraction are two key technologies for constructing discourse-level event timelines, and are also research hotspots in the field of natural language processing. The goal of event extraction is to identify event types and extract argument parameters from unstructured or semi-structured texts. It is one of the most challenging tasks in the field of information extraction, and the precision and recall of discourse-level event extraction are much lower than those of sentence-level event extraction. The discourse-level event extraction task mainly faces two problems: the argument parameter dispersion problem and the multi-event argument parameter assembly problem. Existing deep learning methods find it difficult to understand and model the relationship between sentences and entity mentions in a discourse, and it is also difficult to identify the number of events in a discourse in advance and correctly assign all candidate parameters to different events. Event temporal relation extraction is a type of event relation extraction, and its goal is to identify the order of occurrence of events and classify the temporal relationship of event pairs, such as "After", "Before", etc. The main problem with extracting temporal relations of events at the discourse level is that, compared with sentence-level tasks, discourse-level tasks focus on identifying temporal relations of events in non-adjacent sentences. Using traditional intra-sentence methods will introduce a lot of noise that is of no help in identifying temporal relations due to the long distance between event pairs in the text. In addition, discourse-level tasks also have the problem of inconsistent temporal relations of global events. The classifier's error in judging certain temporal relations will contradict the nature of the temporal relations themselves.
[0005] In summary, the task of extracting discourse-level events has two problems: it is difficult to capture scattered arguments and it is difficult to assemble the arguments of multiple events; the task of extracting discourse-level event temporal relations has two problems: it is difficult to identify the temporal relations of events in non-adjacent sentences and it is difficult to maintain the global consistency of temporal relations. Solving the above problems, improving the accuracy of discourse-level event extraction and discourse-level event temporal relations extraction, and thus generating an accurate discourse-level event timeline, has important theoretical significance and application value. Summary of the invention
[0006] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a method for generating a discourse-level event timeline.
[0007] In order to achieve the purpose of the present invention, we will adopt the following technical solution to implement it.
[0008] A method for generating a discourse-level event timeline comprises the following steps:
[0009] S1, using a discourse-level event extraction method based on heterogeneous graphs to provide event elements;
[0010] S2, using multi-granularity context encoding to extract the temporal relationship of discourse-level events to provide temporal information;
[0011] S3. Generate an event timeline based on the event elements and timing information provided by S1 and S2.
[0012] As a preferred embodiment of the present invention, the method for extracting discourse-level events based on heterogeneous graphs comprises the following steps:
[0013] S1. Build heterogeneous graphs from discourse-level text to model explicit interactions. Process heterogeneous graphs through graph transformation networks to capture implicit interactions of entity mentions and sentences in the discourse, thereby capturing argument parameters scattered in the discourse.
[0014] S2, entity-based constraint tree expansion task, expands leaf nodes according to the predefined argument extraction order to form event records, uses a loss function based on bipartite graph matching to match predicted events with real events, and assembles multi-event arguments.
[0015] As a preferred solution of the present invention, the framework of the discourse-level event extraction method based on heterogeneous graph is divided into two parts: encoder and decoder, wherein:
[0016] The encoder transforms the text sequence structure into a graph structure, models the explicit interaction between sentences and entity mentions, puts the initial graph into the graph transfer network, completes the implicit interaction between sentences and entity mentions, and avoids omissions when extracting argument parameters;
[0017] The decoder identifies event types through sentence embedding, extracts records of each event type, and models the assembly of multi-event argument parameters as a constraint tree expansion task; wherein the leaf nodes of the constraint tree expansion task are candidate argument parameters, which are expanded layer by layer into multiple records, and a loss function based on bipartite graph matching is used to guide model training.
[0018] As a preferred embodiment of the present invention, the encoder comprises a sentence-level encoding layer, a conditional random field layer and a discourse-level encoding layer, wherein:
[0019] The sentence-level encoding layer obtains the context representation of each sentence according to the sentences in the input document;
[0020] The conditional random field layer extracts entity mentions as candidate argument parameters at the sentence level by identifying entities;
[0021] The discourse-level encoding layer captures the global interaction relationship between nodes by constructing a heterogeneous graph with entity mention nodes and sentence nodes, and generates discourse-level representations of sentences and entity mentions.
[0022] As a preferred solution of the present invention, the decoder includes two steps: event type classification and event record extraction, wherein:
[0023] The event type classification step is modeled as a multi-label classification task, and the Sigmoid function is used to determine the event type contained in the document;
[0024] The event record extraction step is to determine whether the candidate entity can become the current argument according to the predefined extraction order of the arguments;
[0025] The condition for entering the event record extraction step from the event type classification step: when the event type classification step determines that the document contains a specific type of event, enter the event record extraction step.
[0026] As a preferred embodiment of the present invention, the method for extracting temporal relations of discourse-level events using multi-granularity context coding comprises the following steps:
[0027] S1. Encode the local context, global context and cross-sentence tense respectively through the local context encoder, global context encoder and cross-sentence tense encoder, map them to the same vector space for connection after encoding, and input them into the Softmax layer for classification task to predict the event temporal relationship of non-adjacent sentences after connection;
[0028] S2. Use the greedy Check-Add process to perform global temporal relationship reasoning, examine one by one in descending probability whether the current edge will introduce conflicts in the temporal graph, and replace the edges with low probability strength based on the probability predicted by the Softmax layer for each relationship class to maintain the global consistency of the temporal relationship.
[0029] As a preferred solution of the present invention, the local context encoder is used to obtain all semantic information of the input event sentence; the global context encoder adopts an attention mechanism to find and encode relevant information in the context; and the cross-sentence temporal encoder adopts an attention mechanism to encode the cross-sentence temporal SDP.
[0030] As a preferred solution of the present invention, the Softmax layer is used as a classifier to predict the time series labels of each pair of events.
[0031] As a preferred embodiment of the present invention, the method for global temporal relationship reasoning includes the following steps:
[0032] S1. Use the probability score of each relationship class in the Softmax layer as the strength of the edge. For a given relationship, add all event vertices to the graph; where all edges are sorted in descending order of probability score.
[0033] S2, use the Timegraph algorithm to determine whether there is a cycle. If so, record the edge to be removed. If not, add it to the graph;
[0034] S3, after all edges have been judged using S2, the removed edges are sorted in ascending order according to the probability score;
[0035] S4, adding edges to the graph one by one using the temporal relationship type with the second highest probability score for the current event;
[0036] S5. After adding the edge, the Timegraph algorithm is used again to determine whether there is a cycle. If so, the time series relationship type with the third highest probability score is used until all the removed edges are added.
[0037] As a preferred solution of the present invention, the Timegraph algorithm specifically includes the following steps:
[0038] ①Start from any unvisited vertex, push it into the stack, and mark it as being visited;
[0039] ② For the vertex at the top of the stack, find an unvisited adjacent vertex. If it exists, push it into the stack and mark it as being visited. If it does not exist, pop the vertex at the top of the stack and mark it as visited.
[0040] ③ Repeat ② until the stack is empty or an adjacent vertex being visited is found;
[0041] ④If an adjacent vertex being visited is found, it means there is a cycle; otherwise, it means there is no cycle;
[0042] ⑤If there are still unvisited vertices, return to ①; otherwise, end the algorithm.
[0043] Beneficial Effects
[0044] First, the present invention solves the two problems of difficulty in capturing scattered argument parameters and difficulty in assembling argument parameters of multiple events according to the discourse-level event extraction method based on heterogeneous graphs;
[0045] 2. The present invention solves the problems of difficulty in judging the temporal relationship of non-adjacent sentence events and difficulty in maintaining the global consistency of the temporal relationship based on a discourse-level event temporal relationship extraction method based on multi-granularity context coding. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 A flowchart of a method for generating a discourse-level event timeline according to the present invention;
[0047] Figure 2 Schematic diagram of cases where candidate argument parameters in the present invention are located in different sentences;
[0048] Figure 3 This is a schematic diagram of a multi-event extraction case in the present invention;
[0049] Figure 4 It is a schematic diagram of the encoder framework in the present invention;
[0050] Figure 5 It is a decoder framework diagram in the present invention;
[0051] Figure 6 This is a schematic diagram of the BIO annotation mode in the present invention;
[0052] Figure 7 is a structural diagram of the heterogeneous graph in the present invention;
[0053] Figure 8 It is a graph transfer network structure diagram in the present invention;
[0054] Fig. 9 This is a schematic diagram of an event record extraction case in the present invention;
[0055] Fig.10 This is a schematic diagram of a case of matching predicted records with event true values in the present invention;
[0056] Fig.11 Schematic diagram of extracting the temporal relationship of discourse-level events in the present invention;
[0057] Fig.12 A general idea diagram for extracting the temporal relationship of discourse-level events in the present invention;
[0058] Fig.13 This is a schematic diagram of an SDP case in the present invention;
[0059] Fig.14 Schematic diagram of cross-sentence tense SDP in the present invention;
[0060] Fig.15 Generate a case diagram for the timeline of events in the present invention. DETAILED DESCRIPTION
[0061] The present invention is further described below in conjunction with specific examples. It should be understood that the following examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.
[0062] As an embodiment of the present invention, Figure 1 As shown in the figure, a method for generating a discourse-level event timeline is as follows:
[0063] The discourse-level event extraction method based on heterogeneous graph is the first step in event timeline generation, and its purpose is to provide event elements for the event timeline.
[0064] The goal of event extraction is to detect the occurrence of events in unstructured or semi-structured texts and extract the arguments of the events. Due to the limitation of data sets, event extraction methods have focused on sentence-level event extraction in the past. However, sentence-level extraction models cannot handle discourse-level event extraction tasks. Specifically, discourse-level event extraction tasks mainly face two problems: (1) the problem of argument parameter dispersion. Figure 2As shown in the figure, the parameters of the start date, pledger and pledgee arguments of the equity pledge event are all in the third sentence, while the parameters of the end date argument and the pledged shares argument are in the fifth sentence. It is difficult for the past event extraction model to extract scattered parameters across sentences. The main reason for this problem is that existing deep learning methods find it difficult to understand and model the relationship between sentences and entity mentions at the discourse level. (2) The problem of assembling parameters of multiple event arguments. Figure 3 As shown in the figure, there are two events in the document, which have the same event type but no obvious text boundary. Existing models usually have difficulty in outputting multiple event records with correct argument parameters. The reason for this problem is that existing models have difficulty in identifying the number of events in the text in advance and correctly assigning all candidate parameters to different events.
[0065] To address the above two issues, the present invention constructs a heterogeneous graph from the discourse-level text, processes the heterogeneous graph through a graph transformation network, captures the interaction of entity mentions and sentences in the discourse, and proposes a discourse-level event extraction method based on the heterogeneous graph.
[0066] The framework of the discourse-level event extraction method based on heterogeneous graph is divided into two parts: encoder and decoder. Figure 4 As shown in , the encoder part includes a sentence-level encoding layer, a conditional random field layer, and a discourse-level encoding layer. First, the sentences in the document are input into the sentence-level encoding layer to obtain the contextual representation of each sentence. This part uses Transformer as the context encoder. Then, the entity is identified through the conditional random field, and the entity mentions are extracted at the sentence level as candidate argument parameters. Finally, the discourse-level encoding layer captures the global interaction relationship between nodes by constructing a heterogeneous graph with entity mention nodes and sentence nodes, and generates discourse-level representations of sentences and entity mentions. The decoder part is shown in Figure 5 As shown in the figure, the decoder part includes two steps: event type classification and event record extraction. Since multiple different types of events may be expressed, event type classification can be modeled as a multi-label classification task, and the Sigmoid function is used to determine the event type contained in the document. When the classifier determines that the document contains a specific type of event, it enters the event record extraction step and determines whether the candidate entity can become the current argument based on the predefined extraction order of the argument. Based on this idea, each path from the root node to the leaf node will become an event record.
[0067] The main reason why the argument parameter dispersion problem is difficult to solve is that previous deep learning methods have difficulty understanding and modeling the relationships between sentences and entity mentions at the discourse level. Therefore, the present invention converts the input of the text sequence structure into a graph structure, uses sentences and entity mentions as nodes of the graph structure, and uses four different edges to model the interaction between sentences, the interaction between sentences and entity mentions in sentences, the interaction between cross-sentence entity mentions, and the interaction between entity mentions within sentences, so that the model has the ability to extract argument parameters across sentences. However, the sentence nodes in the heterogeneous graph constructed by the GIT model are fully connected by default. Although this method allows all sentence nodes to aggregate information from other sentence nodes, it also brings a large number of invalid edges, which introduces noise to the representation of sentence nodes. Figure 2 The sentence node of the third sentence in the sentence embedding vector can generate a valid sentence embedding vector by only aggregating the sentence node information of the fifth sentence, while aggregating all the sentence node information will bring additional noise to the embedding vector. In addition, the text often contains a large number of implicit relationships between sentences and entity mentions, and the generated initial graph often cannot fully capture all entity mentions and the relationship between sentences. Therefore, only sentence nodes with the same entity mention and adjacent sentence nodes are connected, and the graph transfer network is used to complete the possible valid connections in the generated initial graph structure and learn the representation of nodes in the new graph.
[0068] The first task that the encoder needs to handle is to convert text data into mathematical representations so that the semantic information of words can be processed by the neural network. Formally, we can define the task of discourse-level event extraction as follows: Given a document consisting of several sentences Each sentence s i Contains a sequence of words Define the predefined event type sets as The argument set is A. The discourse-level event extraction task aims to extract one or more structured events. Each of these events Contains several parameters Filled arguments Where a∈A. k is the number of events contained in the document, and n is the type of each event. The sentence-level encoding layer uses Transformer to convert sentences Encoded as a sequence of vectors As shown in formula (3-1), Transformer is different from traditional Word2vec. It can aggregate contextual information to generate dynamic word embedding vectors. Compared with static word embedding vectors, it contains richer semantic information and is currently one of the most widely used models in generating word embedding vector tasks.
[0069] {g1,…,g |s|}=Transformer({w1,…,w |s|}) (3-1)
[0070] The word embedding w j It contains word vector encoding and position encoding information.
[0071] Event argument parameters are usually nouns used to express an entity, such as Figure 3 In the event argument table on the right side, the parameters of the shareholder argument are name entities such as "CF" and "PJ", while the parameters of the start and end time are time entities such as "October 30, 2018". There will be a large number of entity mentions in the discourse, and these entity mentions will become parameters of different arguments in multiple events, so after sentence-level encoding, it is necessary to extract the entity mentions in all sentences as candidate argument parameters. The present invention defines the entity extraction process as a sequence labeling task of the BIO mode, and uses conditional random fields to identify entity mentions. The BIO mode uses three types of tags, B, I, and O, to label elements. B represents the beginning part, I represents the middle part, and O represents the other parts. Figure 6 This is an example of using the BIO mode to annotate entity mentions. The loss function of the conditional random field during training is shown in formula (3-2), where s is a sentence and y s is the golden label sequence of s, and the Viterbi algorithm is used to decode the label sequence with the greatest probability.
[0072]
[0073] The description of an event in a text may span multiple sentences in a document, which means that the entity mentions of the event are also scattered in different sentences. Modeling and capturing the contextual interactions of these entity mentions is the key to text-level event extraction. With the help of a graph structure, this paper constructs a heterogeneous graph containing entity mention nodes and sentence nodes, denoted as graph In the figure The edges between the nodes explicitly model the interactions between entity mentions and sentences.
[0074] First, initialize the node embedding. The initial node embedding of the entity mention node e is generated by the mean of all word representations contained in the entity mention, as shown in formula (3-3). The initial node embedding of the sentence node S is generated by the Maxpooling operation of the word representation contained in the sentence plus the sentence position embedding, as shown in formula (3-4).
[0075]
[0076]
[0077] The present invention captures the interaction between sentences and entity mentions through four types of edges between nodes in the graph. Figure 7 Introduce the functions of the four edges respectively.
[0078] (1) Cross-sentence entity-entity edges. Entity mentions of the same entity in different sentences are connected by cross-sentence entity-entity edges, such as Figure 7 As shown by the blue dashed line in . An entity in a text usually has multiple mentions across sentences, so we use cross-sentence entity-entity edges to capture all contextual relationships of a specific entity mention, which helps to extract events with a large text span from a global perspective. (2) Intra-sentence entity-entity edges. Connect different entity mentions in the same sentence, such as Figure 7 As shown by the blue solid line in the middle, the presence of multiple entity mentions in the same sentence indicates that these mentions are likely to be related to the same event. This helps capture the local entity mention relationship within a sentence. (3) Entity-sentence edge. Connect a sentence with the entity mention node it contains, such as Figure 7 As shown by the red solid line in the figure, the entity-sentence edge models the local context of the entity mention in the sentence. (4) Sentence-sentence edge. The sentence-sentence edge is shown by the red dotted line in the figure. In the GIT model, all sentence nodes are fully connected. Although fully connecting all sentence nodes can capture possible sentence interactions, it also brings a large number of invalid edges, which brings noise to the node representation. Therefore, the present invention only connects sentence nodes that have the same entity mention, captures the explicit interaction relationship between sentences, and the implicit interaction relationship is completed by the graph transfer network.
[0079] After the heterogeneous graph is constructed, the GIT model uses a multi-layer graph convolutional network to aggregate global node features and generate node embeddings. However, although graph convolutional networks are widely used to learn node representations on graphs, the premise is that the graphs are fixed and isomorphic. When there are errors in the graph itself or when learning node representations on a heterogeneous graph composed of different types of nodes and edges, the node representation generated by the graph convolutional network is poor. In the task of generating heterogeneous graphs from text, the implicit interactive relationships between sentences and entity mentions in the text are difficult to model in the graph, resulting in the initial graph often being inaccurate. Therefore, the present invention uses a graph transfer network to complete the interactive information and extract features. The graph transfer network can identify useful connections between unconnected nodes in the original graph, and use the adjacency matrix as a candidate to find a new graph structure, and learn effective node representations in the new graph to avoid the loss of implicit relationships between nodes. Figure 8 As shown, the adjacency matrices Q1 and Q2 in the adjacency matrix set A are softly selected through 1x1 convolution, and a new meta-path graph A is generated through matrix multiplication of the two selected adjacency matrices and l , Figure 8The blue line in the figure is the initial meta-path, and the red line is the new meta-path. After the new meta-path is generated, a graph convolution operation is performed on the new graph structure. The graph convolution operation of obtaining the node u of the l+1 layer from the node u of the lth layer is shown in formula (3-5).
[0080]
[0081] where k represents different types of edges, is a trainable parameter, N k (u) represents the neighboring nodes connected between node u and the kth edge, C u,k is a normalization constant. The hidden state h of the heterogeneous graph node u u As shown in formula (3-6).
[0082]
[0083] in is the initial state of node u, and L is the number of graph convolution layers.
[0084] Finally, we get the sentence embedding matrix through graph convolution and the entity embedding matrix Since the same entity may have multiple mentions, and multiple mention nodes of the same entity generate multiple embedding matrices in the graph convolution, the entity embedding matrix E of the i-th entity is i It is calculated by the mean of the embeddings of the mentioned nodes, namely E i =Mean({h j} j∈Mention(i) ), where Mention(i) represents the set of mentions of the i-th entity, h j The embedding vector representing the mention of the jth entity in the set. By combining sentence-level and discourse-level encoding, the contextual information and interaction information of sentences and entity mentions can be encoded in the embedding matrix.
[0085] The difficulty of multi-event argument parameter assembly problem lies in the fact that it is difficult for the model to identify the number of events in the text in advance and correctly assign all candidate parameters to different events. Figure 3As shown, the discourse-level event extraction task can be formally viewed as a filling task of multiple event tables, that is, filling candidate entities into predefined event tables. The main function of the decoder is to process the encoded sentence embedding and entity mention embedding, use the two embeddings to perform the event table filling task, and assign the correct entity as the argument parameter for each type of triggered event. The present invention models the multi-event argument assembly problem as an entity-based constraint tree expansion task. For each triggered event type, the leaf node is expanded according to the predefined argument extraction order, and each expanded node is regarded as a binary classification problem. Finally, the events contained in the document are restored according to the path. At the same time, considering that the number of predicted events is often not equal to the number of real events, it is necessary to design a matching loss function that can effectively assign predicted events to real events to guide the training of the model. Therefore, the present invention models the matching problem between predicted events and real events as a bipartite graph matching problem, and proposes a loss function based on the bipartite graph matching problem to optimize the training effect.
[0086] Each discourse may contain any number of different types of events, which means that discourse-level event type classification is a multi-label classification problem. The present invention uses the sentence embedding matrix S to classify event types, as shown in formulas (3-7) and (3-8).
[0087]
[0088]
[0089] in and are all trainable parameters, T represents the number of possible event types. MultiHead represents a standard multi-head attention mechanism with Q (Query), K (Key), and V (Value). Therefore, the event type classification loss is shown in (3-9).
[0090]
[0091] in For gold labels.
[0092] After a certain event type is triggered, the event record extraction step of this type of event is entered. Event record extraction can be modeled as a constraint tree expansion task. Fig. 9As shown in the figure, taking the equity increase event as an example, the equity holder argument is first extracted, and then the trading shares, start time, end time, etc. are extracted in sequence. Starting from the root node, the tree is expanded by predicting parameters in sequence. Since the argument parameters of the event may involve multiple qualified entities, the current node will expand multiple branches during the extraction process. For example, there are two qualified entities A and B for the shareholders, and two qualified entities C and D for the trading shares. This is because there may be multiple different event records for the same event type in the discourse, and for the same event record, the model may also have multiple reasonable prediction results. This branching operation is modeled as a multi-label classification task. As mentioned above, each path from the root node to the leaf node is identified as an event record.
[0093] The main problem in training the event record extraction model based on the constraint tree expansion task is how to assign the predicted m event records to the k event records in the true value, so as to determine the gap between the predicted results and the true value.
[0094] like Fig.10 As shown in the figure, the left side is the event record predicted by the model, and the right side is the event truth. There are only two records in the text, event 1 and event 2, and the model cannot predict the number of true value events in advance, such as Fig.10 At this time, there are four event records. First, it is necessary to determine whether the event record is valid. In the present invention, the event key arguments marked in the Doc2EDAG model are used as the basis for determining whether the predicted event is valid. For example, for an equity increase event, if the two key arguments of the shareholder and the traded shares do not exist, the extracted record should be considered invalid. After determining the validity of the event, it is necessary to match the event record on the left with the true value event on the right according to the degree of matching, that is, to solve the optimal matching problem of the bipartite graph.
[0095] Formally, the m event records predicted by the model can be defined as The true value of k events can be defined as Y = (Y1, Y2, ..., Y k ), the i-th event in the predicted event record can be defined as in represents the jth argument parameter predicted in the i-th event. The i-th event in the true value event can be defined as in is the jth argument parameter in the ith truth event. (n and l are With Y i The optimal binary match between the two groups of events is found by using the number of argument parameters contained in , and then the loss function is defined according to the gap between the predicted event and the true value event when the optimal binary match is achieved. Specifically, the following steps are included:
[0096] (1) For each prediction record Calculate it and all true value records Y i The similarity between As shown in (3-10), for The p-th argument parameter in Y i The p-th argument parameter is included.
[0097]
[0098] (2) Use the Hungarian algorithm to find the optimal matching σ(i) on an m×k bipartite graph, where each node represents a predicted record or a true value record, and the weight of each edge is Specifically, the steps of the Hungarian algorithm are as follows:
[0099] ① For each row, subtract the minimum element of the row from all the elements in the row so that each row has at least one zero element.
[0100] ② For each column, subtract the minimum element of the column from all the elements in the column so that each column has at least one zero element.
[0101] ③ Cover all zero elements with as few horizontal and vertical lines as possible.
[0102] ④If the number of covering lines is equal to the order of the matrix (i.e. min(m,k)), an optimal allocation solution has been found; otherwise, proceed to the next step.
[0103] ⑤ Find the minimum value among the uncovered elements, subtract it from all uncovered elements, and add it to all elements at the intersection.
[0104] ⑥ Return to ③ and repeat until an optimal allocation solution is found.
[0105] (3) Through matching loss Indicates whether each argument parameter in the predicted record is the same as the argument parameter in the same position in the matching true value record. If there is no matching true value record, use the all-zero vector as the target argument parameter vector. As shown in formula (3-11), is a binary label, indicating Whether the p-th argument parameter in is optimally matched to Y i The p-th argument parameter in σ(i) is the same (if there is no matching truth value record, then is 0)
[0106]
[0107] When the model is trained, the final loss function is the loss generated by the conditional random field layer Event type classification loss Bipartite Graph Matching Loss in Event Record Extraction The result of adding the three weights is shown in (3-12).
[0108]
[0109] Among them, λ1, λ2, and λ3 are hyperparameters.
[0110] As an embodiment of the present invention, Table 1 compares the experimental results of the discourse-level event extraction method based on heterogeneous graph and the best baseline GIT under the same discourse-level text case, wherein Table 1(a) is a discourse case, which contains 15 sentences, two equity increase events and one equity reduction event, and the bold text in the text is the argument parameter of the equity reduction 1 event, the underlined text is the argument parameter of the equity increase 1 event, and the italic text is the argument parameter of the equity increase 2 event. Table 1(b) and Table 1(c) are the extraction effects of the discourse-level event extraction method based on heterogeneous graph and the GIT model on the discourse, respectively. It can be found from the above two tables that for the two equity increase events, the extraction effects of GIT and the discourse-level event extraction method based on heterogeneous graph are consistent, and the argument parameters of the events are correctly extracted and assembled; for the equity reduction event, the discourse-level event extraction method based on heterogeneous graph correctly extracts the final shareholding argument, while the GIT model fails to extract the final shareholding argument. "6740000 shares" comes from the eighth sentence in the text, while "PH", "January 7, 2009", and "4000000 shares" come from the fifth sentence in the text, with two sentences in between, and there is no common entity mention between the fifth and eighth sentences. Because the fully connected heterogeneous graph in GIT introduces too many invalid edges and cannot capture the scattered argument parameters with implicit interaction relationships, the GTNs-based heterogeneous graph constructed in the text-level event extraction method based on heterogeneous graphs can identify useful connections between unconnected nodes in the original graph, complete implicit interaction relationships and extract features, thereby effectively capturing the argument parameters scattered in the text.
[0111] Table 1 Examples of discourse-level event extraction
[0112] Table 1(a) Discourse
[0113]
[0114] Table 1(b) Extraction results of discourse-level event extraction method based on heterogeneous graph
[0115]
[0116] Table 1 (c) GIT extraction results
[0117]
[0118] The method of extracting event temporal relations at the discourse level with multi-granularity context encoding is the second step in event timeline generation, which aims to provide temporal information for the event timeline. Event temporal relation extraction is a basic task in the field of event relation extraction. Event temporal relation extraction is crucial for natural language understanding and is usually combined with event extraction tasks to complete various downstream applications, such as summarization, question answering, and knowledge graph construction.
[0119] The temporal relationships of events in texts include the following three categories: event-event temporal relationships, event-temporal expression temporal relationships, and event-document creation time temporal relationships. This invention mainly studies the method of extracting event-event temporal relationships. Most of the early methods for extracting temporal relationships between events are based on statistical machine learning. However, this type of method has difficulty in identifying and representing complex, implicit event temporal relationships. In recent years, neural network-based methods combined with large-scale pre-trained language models have greatly improved the performance of event temporal relationship extraction tasks. These methods are effective for sentence-level tasks. However, in discourse-level tasks, a large number of events are scattered in different sentences, and the textual distance between events is usually long. This brings two major difficulties to the extraction of discourse-level event temporal relationships: (1) The problem of judging the event temporal relationship between non-adjacent sentences. For example Fig.11 As shown in the figure, the text contains four events: e1 (moving out), e2 (narration), e3 (walking the dog) and e4 (killing). There are three sentences between events e1 and e2. When using traditional neural network methods to input all the information between e1 and e2 to identify the temporal relationship between e1 and e2, a large amount of noise that is of no help in identifying the temporal relationship will be introduced due to the long text distance between the events. In fact, the model only needs to capture the context related to these two events. How to select appropriate context information so that the model will not miss information that is helpful for the temporal relationship and introduce as little irrelevant information as possible is the key to the problem of judging the temporal relationship of non-adjacent sentence events; (2) The problem of global consistency of temporal relationships. The problem of global consistency of temporal relationships means that the results of event temporal relationship extraction at the text level must follow the constraints of the temporal relationship itself. For example Fig.11 If the neural network classifier classifies the relationship between e1 and e2, the relationship between e2 and e3, and the relationship between e3 and e4 as "After", according to the transitivity of the temporal relationship, the relationship between e1 and e4 should be "After". If the classifier does not classify it as "After", there is a global inconsistency problem in the temporal relationship, and the classifier makes incorrect judgments on some temporal relationships.
[0120] In view of the above two problems, the present invention extracts a variety of context information from the discourse-level text and proposes discourse-level event temporal relationship extraction based on multi-granularity context encoding. The discourse-level event temporal relationship encoded by multi-granularity context is encoded by three encoders: local context encoder, global context encoder and cross-sentence temporal encoder. After connection, it is input into the Softmax layer to perform the classification task and obtain the prediction result. Finally, the greedy Check-Add process is used to perform global temporal relationship reasoning to maintain the global consistency of the temporal relationship.
[0121] like Fig.12 As shown in the figure, the discourse-level event temporal relationship framework of multi-granularity context encoding is divided into local context encoder, global context encoder, cross-sentence temporal encoder and classifier. The local context encoder uses BERT to process the input event sentence, extracts the information of the event sentence itself, and encodes the local context of the sentence containing the specific event into a vector. The global context encoder first identifies the entity mentions in the input event sentence, selects the sentences with the same entity mentions, then selects the sentences as the global context information according to the similarity, and finally fuses the current sentence with the global context information to generate the global context encoding. The cross-sentence temporal encoder uses SUTime to extract the temporal expressions of all sentences between event sentences, adds the temporal expressions to the shortest dependency path, and then uses the attention mechanism as the encoder, and uses Maxpooling to output the feature vector. Finally, the vectors output by the three encoders are connected and sent to the Softmax layer to predict the temporal labels of the event pairs. After the time label prediction of the event pairs in the text is completed, the global temporal relationship reasoning mechanism is used to maintain the global consistency of the temporal relationship.
[0122] The main reason why it is difficult to judge the temporal relationship of events in non-adjacent sentences is that the judgment of the temporal relationship of events in a pair of non-adjacent sentences cannot rely solely on the information of the sentences themselves, but also requires the model to understand the global information of the discourse. Temporal relationship extraction is different from event extraction tasks. Since event extraction tasks need to extract various arguments of events, in order to avoid omissions in argument capture, context-related information needs to be fully encoded. However, the judgment of event temporal relationships usually requires a variety of fine-grained information, such as tense expressions, expression order, event types, etc. in the context. Comprehensive encoding of event-related information will introduce a lot of information that is invalid for judging event temporal relationships. For example Fig.11 To judge the temporal relationship between e1 and e4, local context information of the statements where e1 and e4 events are located, global context information of the statements related to e1 and e4 events, and tense expression information mentioned between e1 and e4 statements are required. Therefore, the present invention divides the information that helps to judge the temporal relationship into three categories: local context, global context, and cross-sentence tense. The three types of information are encoded separately and mapped to the same vector space for connection, and then the relationship labels of the event pairs are predicted through the Softmax layer.
[0123] The local context encoder is used to obtain all the semantic information of the input event sentence. This module uses BERT as the encoder, which can effectively capture the long-distance dependencies and semantic features in natural language. The model uses [E1][E1 / ][E2][E2 / ] to mark the start and end of the event. The encoder input s sen As shown in formula (4-1).
[0124]
[0125] [CLS] is used to mark the beginning of the input information, and [SEP] is used to indicate the boundary between two statements and is placed at the end of the statement. m} and s2={t1,t2,…,t n} represents an event sentence, m and n are the lengths of two event sentences respectively. e1={w i ,…,w j} and e2={t k ,…,t l} are two events contained in the sentence, where i≥1,j≤m and k≥1,l≤n. The output vector sequence of BERT can be expressed as formula (4-2).
[0126] {e [CLS] ,…,e [E1] ,…,e [SEP] ,…, (4-2)
[0127] where e [CLS] is a vector containing the semantic information of the event sentence, which is used as the output of the local context encoder loc .
[0128] In the task of event temporal relationship extraction, when two events exist in the same sentence or adjacent sentences, it is relatively easy to distinguish the temporal relationship between them. However, at the discourse level, it is necessary to process more event temporal relationships involving non-adjacent sentences, which puts higher requirements on the extraction method. The event temporal relationship extraction of non-adjacent sentences requires not only the information of the sentence itself, but also the selection of relevant contextual information in the global discourse. The global contextual information can establish a connection between two sentences that are physically separated in the discourse and supplement the potential semantic features. In order to solve the above problems, the present invention establishes a global context encoder to find and encode relevant information in the context.
[0129] The first step of the global context encoder is to establish a context information selection mechanism to select the sentences that are most relevant to the two event sentences. Context information selection includes two steps: entity-level related sentence selection and semantic similarity calculation. Entity-level related sentence selection is inspired by heterogeneous graphs. Sentences containing the same entity mentions are usually closely related in the text. The Stanford CoreNLP tool is used to identify entity mentions in the text and select sentences with the same entity mentions as the two event sentences for semantic similarity calculation (if there is no identical entity mention, the semantic similarity calculation is performed directly). At the semantic level, the two input event sentences are first encoded using BERT, using [CLS] as the semantic vector. The semantic vectors of the event sentences are represented here as e-s1 and e-s2, respectively. Then all sentences in the text are encoded in this way, denoted as e-con i , where i is the number of sentences in the text. Finally, calculate e-con i With e-s1, e-con i The cosine similarity with e-s2, select the two sentences with the highest scores con_1 = {c 11 ,…c 1h} and con_2={c 21 ,…c 2g} as the global context information, where h and g are the lengths of statements con_1 and con_2 respectively.
[0130] After the global context information is determined, con_1 and con_2 are connected according to the input form of BERT, as shown in formula (4-3).
[0131] s con ={[CLS],c 11 ,…,c 1h ,[SEP],c 21 ,…,c 2g ,[SEP]} (4-3)
[0132] Will s con After input into BERT, the output vector sequence e con As the semantic vector sequence of global context, as shown in formula (4-4).
[0133] e con ={e [CLS] ,e 11 ,…,e 1h ,e [SEP] ,e 21 ,…,e 2g ,e [SEP]} (4-4)
[0134] BERT can extract all semantic information in the entire sentence. At the same time, if there is more specific information about the event to be extracted, BERT can capture contextual information of different dimensions hierarchically. con After that, e sen e in [E1] With e [E2] With the global context semantic vector sequence e con concatenate, as a vector e cs , as shown in formula (4-5).
[0135] e cs ={e [E1] ,e [E2] ,e con} (4-5)
[0136] Since the attention mechanism has the ability to capture the correlation between words, self-attention is used to encode the vector e cs , and extract the specific context feature vector e related to the event through max pooling glo , as shown in formula (4-6).
[0137] e glo =Maxpooling(Attention(e cs ,e cs ,e cs )) (4-6)
[0138] In addition to local context and global context information, temporal expressions are another explicit clue in the text that indicates event relationships. Therefore, SDP is modified by adding temporal expressions. SDP refers to the shortest directed path between two words connected by a dependency relationship in a sentence. Fig.13 For the dependency tree of the sentence "Andrei tells about an incident that happened when he and his family lived in Moscow", SDP walks along the directed edges of the dependency tree to the root node, where the directed edges represent the grammatical relationship between words in the sentence. SDP is often used in event relationship extraction research to capture the most relevant grammatical and semantic information between two words in a sentence.
[0139] However, the premise of using SDP for relation extraction is that the words are in the same sentence. SDP is used in cross-sentence event temporal relation extraction, and the concept of virtual root is proposed, that is, two SDPs in adjacent sentences are connected by virtual root. However, this approach cannot be applied to relation extraction of non-adjacent sentences, because virtual root will ignore the useful clues between two scattered event sentences.
[0140] Therefore, we use the temporal expression annotation tool SUtime to extract all temporal expressions from the sentences between the two event sentences s1 and s2, and add the temporal expressions to the path connecting the two original SDPs to form a cross-sentence temporal SDP. If there are multiple temporal expressions, they are added to the path in text order. Fig.11 Taking the two events e1 and e4 in the example, SUTime is used to extract the tense expression "last year" from the text between the two event sentences, and then the tense expression is inserted into the two SDPs according to the sentence order to form a cross-sentence tense SDP, such as Fig.14 shown.
[0141] The attention mechanism is used as the encoder to overcome the problem that BiLSTM cannot effectively capture long-distance dependencies. Specifically, at the attention input, each word in the cross-sentence temporal SDP is represented by GloVe embedding. T ={v1,v2,…v k-1 ,v k} represents the embedding sequence of cross-sentence tense SDP, where v i is the embedding representation of each word in the cross-sentence temporal SDP, and k is the maximum length of the cross-sentence temporal SDP. At the attention output, a maximum pooling operation is used to extract the feature vector e of the cross-sentence temporal SDP. tem , as shown in formula (4-7).
[0142] e tem =Maxpooling(Attention(e T ,e T ,e T )) (4-7)
[0143] The local context vector e loc , global context vector e glo and the cross-sentence tense vector e tem Mapped to a common vector space to achieve full connection. Then, they are fed into the Softmax layer to predict the time series label of each pair of events, as shown in formula (4-8).
[0144]
[0145] Where W is the weight matrix, b is the bias, During training, the Adam optimizer with a learning rate of 3e-5 is used to minimize the cross entropy loss, and the model parameters are updated using back propagation.
[0146] The reason why the global consistency of temporal relations is difficult to maintain is that the results of multi-event temporal relations extraction should follow the transitivity of temporal relations from a global perspective. However, existing deep learning models all identify temporal relations in units of event pairs, which makes it difficult to ensure global consistency, and the extraction results are often contradictory. Therefore, the present invention proposes a greedy Check-Add process to perform global temporal relation reasoning, and examines whether the current edge will introduce conflicts in the temporal graph one by one in descending order of probability score, and replaces the edges with low probability strength in combination with the probability predicted by the Softmax layer for each relationship class to maintain the global consistency of the temporal relations.
[0147] The output of the text-level event temporal relation extraction can be viewed as an event time graph. Where V is the set of nodes in the graph, that is, the set of events, and E is the edge in the graph, that is, the temporal relationship between events. The event time graph is a directed acyclic graph used to describe the temporal relationship between events. The word "directed" means the temporal relationship between events, and the word "acyclic" means that there is no time loop between events. A time loop means that the temporal relationship between events forms a closed loop. Specifically, if there is a time loop in the event time graph, there will be a global inconsistency problem in the temporal relationship. For example Fig.11 In the example above, if the model classifies (e1, e2) and (e2, e3) as "After" relations and (e1, e3) as "Before" relations, then at least one edge in the path forming the loop violates the transitivity principle. Therefore, after the prediction of the temporal relations of discourse-level events is completed, the global temporal relations reasoning link is required to maintain the global consistency of the temporal relations.
[0148] In the reasoning stage, the present invention proposes a greedy Check-Add process to construct a globally consistent event time graph. The specific approach is shown in Algorithm 1 and is divided into the following steps:
[0149] (1) Using the probability score of each relationship class in the Softmax layer as the strength of the edge, for a given relationship (Before or After), add all event vertices to the graph. Sort all edges in descending order of probability score.
[0150] (2) Use the Timegraph algorithm to determine whether there is a cycle. If so, record the edge being removed. If not, add it to the graph. The Timegraph algorithm is an algorithm for detecting whether there is a cycle in a directed graph. It is based on the idea of depth-first search and specifically includes the following steps:
[0151] ①Start from any unvisited vertex, push it into the stack, and mark it as being visited.
[0152] ② For the vertex at the top of the stack, find an unvisited adjacent vertex. If it exists, push it into the stack and mark it as being visited; if it does not exist, pop the top vertex of the stack and mark it as visited.
[0153] ③ Repeat ② until the stack is empty or an adjacent vertex being visited is found.
[0154] ④If an adjacent vertex being visited is found, it means there is a cycle; otherwise, it means there is no cycle.
[0155] ⑤If there are still unvisited vertices, return to ①; otherwise, end the algorithm.
[0156] (3) After all edges have been judged using the above method, the removed edges are sorted in ascending order of probability score. The temporal relationship type with the second highest probability score for the current event is used to add edges to the graph one by one. If it is not a Before and After relationship, no loop is considered. Although Includes, Is_Included, and Simultaneous can also generate conflicts in the Before / After graph, the resolution of such conflicts depends largely on whether the event information in the dataset itself is complete and standardized, and in actual situations, the quality of the text is usually difficult to control.
[0157] (4) After adding the edge, the Timegraph algorithm is used again to determine whether there is a cycle. If so, the temporal relationship type with the third highest probability score is used until all the removed edges are added.
[0158]
[0159] As an embodiment of the present invention, the event timeline generates an example
[0160] In the financial field, there are a large number of unstructured or semi-structured financial reports, news reports, company announcements, etc. In order to understand corporate dynamics and obtain investment advice, financial practitioners usually need to spend a lot of time reading and understanding texts. However, these reports are numerous, the text information is complex, and the text is long. Manual reading and arranging them into structured information is extremely costly and inefficient.
[0161] We will use a text in a financial institution investigation report as an example to generate an event timeline. As shown in Table 2, there are four equity increase events, one equity pledge event, and one equity freeze event. First, we extract events from the text using a text-level event extraction method based on a heterogeneous graph. Then, since the present invention uses an English dataset for training, we manually translate the cases into English and then label the events. Finally, we use the proposed multi-granularity context-encoded text-level event temporal relationship to extract the temporal relationship between events, and generate the following: Fig.15The event timeline shown. Through the event timeline, practitioners can quickly understand the development status of the enterprise and provide a basis for decision-making.
[0162] Table 2 Example texts for generating discourse-level event timeline
[0163]
[0164] The technical solution of the present invention is described in detail above in conjunction with the embodiments / drawings, but the present invention is not limited to the above technical solution. For ordinary technicians in this technical field, after knowing the contents recorded in the present invention, they can make several equivalent changes and substitutions without departing from the principle of the present invention. These equivalent changes and substitutions should also be regarded as belonging to the protection scope of the present invention.
Claims
1. A method for generating a discourse-level event timeline, characterized in that: The steps include: S1, using a discourse-level event extraction method based on heterogeneous graphs to provide event elements; S2, using multi-granularity context encoding to extract the temporal relationship of discourse-level events to provide temporal information; S3. Generate an event timeline based on the event elements and timing information provided by S1 and S2.
2. A method for generating a discourse-level event timeline according to claim 1, characterized in that: The method for extracting discourse-level events based on heterogeneous graphs comprises the following steps: S1. The step of capturing scattered argument parameters is to construct a heterogeneous graph from the discourse-level text to model explicit interactions, process the heterogeneous graph through a graph transformation network, capture the implicit interactions of entity mentions and sentences in the discourse, and thus extract the argument parameters scattered in the discourse; S2, the step of assembling multi-event arguments, based on the entity constraint tree expansion task, expands the leaf nodes to form event records according to the predefined argument extraction order, and uses the loss function based on bipartite graph matching to match the predicted events with the real events.
3. A method for generating a discourse-level event timeline according to claim 2, characterized in that: The framework of the heterogeneous graph-based discourse-level event extraction method is divided into two parts: an encoder and a decoder, where: The encoder transforms the text sequence structure into a graph structure, models the explicit interaction between sentences and entity mentions, puts the initial graph into the graph transfer network, completes the implicit interaction between sentences and entity mentions, and avoids omissions when extracting argument parameters; The decoder identifies event types through sentence embedding, extracts records of each event type, and models the assembly of multi-event argument parameters as a constraint tree expansion task; wherein the leaf nodes of the constraint tree expansion task are candidate argument parameters, which are expanded layer by layer into multiple records, and a loss function based on bipartite graph matching is used to guide model training.
4. A method for generating a discourse-level event timeline according to claim 3, characterized in that: The encoder comprises a sentence-level encoding layer, a conditional random field layer and a discourse-level encoding layer, wherein: The sentence-level encoding layer obtains the context representation of each sentence according to the sentences in the input document; The conditional random field layer extracts entity mentions as candidate argument parameters at the sentence level by identifying entities; The discourse-level encoding layer captures the global interaction relationship between nodes by constructing a heterogeneous graph with entity mention nodes and sentence nodes, and generates discourse-level representations of sentences and entity mentions.
5. The method for generating a discourse-level event timeline according to claim 3, characterized in that: The decoder includes two steps: event type classification and event record extraction, where: The event type classification step is modeled as a multi-label classification task, and the Sigmoid function is used to determine the event type contained in the document; The event record extraction step is to determine whether the candidate entity can become the current argument according to the predefined extraction order of the arguments; The condition for entering the event record extraction step from the event type classification step: when the event type classification step determines that the document contains a specific type of event, enter the event record extraction step.
6. A method for generating a discourse-level event timeline according to claim 1, characterized in that: The method for extracting the temporal relationship of discourse-level events using multi-granularity context coding comprises the following steps: S1. Encode the local context, global context and cross-sentence tense respectively through the local context encoder, global context encoder and cross-sentence tense encoder, map them to the same vector space for connection after encoding, and input them into the Softmax layer for classification task to predict the event temporal relationship of non-adjacent sentences after connection; S2. Use the greedy Check-Add process to perform global temporal relationship reasoning, examine one by one in descending probability to see whether the current edge will introduce conflicts in the temporal graph, and replace the edges with low probability strength with the predicted probability of each relationship class in combination with the Softmax layer to maintain the global consistency of the temporal relationship.
7. A method for generating a discourse-level event timeline according to claim 6, characterized in that: The local context encoder is used to obtain all semantic information of the input event sentence; The global context encoder uses an attention mechanism to find and encode relevant information in the context; The cross-sentence temporal encoder adopts an attention mechanism to encode the cross-sentence temporal SDP.
8. The method for generating a discourse-level event timeline according to claim 6, characterized in that: The Softmax layer is used as a classifier to predict the time series labels of each pair of events.
9. The method for generating a discourse-level event timeline according to claim 6, characterized in that: The method for global temporal relationship reasoning comprises the following steps: S1. Use the probability score of each relationship class in the Softmax layer as the strength of the edge. For a given relationship, add all event vertices to the graph; where all edges are sorted in descending order of probability score. S2, use the Timegraph algorithm to determine whether there is a cycle. If so, record the edge to be removed. If not, add it to the graph; S3, after all edges have been judged using S2, the removed edges are sorted in ascending order according to the probability score; S4, adding edges to the graph one by one using the temporal relationship type with the second highest probability score for the current event; S5. After adding the edge, the Timegraph algorithm is used again to determine whether there is a cycle. If so, the time series relationship type with the third highest probability score is used until all the removed edges are added.
10. The method for generating a discourse-level event timeline according to claim 9, characterized in that: The Timegraph algorithm specifically includes the following steps: ①Start from any unvisited vertex, push it into the stack, and mark it as being visited; ② For the vertex at the top of the stack, find an unvisited adjacent vertex. If it exists, push it into the stack and mark it as being visited. If it does not exist, pop the vertex at the top of the stack and mark it as visited. ③ Repeat ② until the stack is empty or an adjacent vertex being visited is found; ④If an adjacent vertex being visited is found, it means there is a cycle; otherwise, it means there is no cycle; ⑤If there are still unvisited vertices, return to ①; otherwise, end the algorithm.
Citation Information
Patent Citations
Multi-task chapter-level event extraction method based on multi-headed self-attention mechanism
CN113761936A
Method, system and equipment for extracting chapter-level events
CN114168738A
Joint learning-based closed domain chapter-level event extraction method
CN117743600A
System and method for timeline visualization and interaction
US20120331378A1
Cited By
Event argument extraction method for military field
CN119886125A
An event-oriented argument extraction method for military field
CN119886125B
A unit business record set generation method and device based on a large model
CN122692011A
A unit business record set generation method and device based on a large model
CN122692011B
