Event graph intelligent construction and analysis method and device based on BERT model

Abstract

The application relates to the field of information technology, in particular to an event graph intelligent construction and analysis method and device based on a BERT model, wherein the method comprises the following steps: extracting event information from target network text; converting the extracted events corresponding to the event information into extracted event graphs based on a preset event graph, wherein the extracted events reflecting the same real event are fused into one event graph by event fusion; and based on a pre-trained BERT model, discovering the same points of different events, predicting event relationships, constructing a complete event context, and determining the evolution mode and law of the events by using a knowledge graph and event relationship prediction. The application can quickly and effectively obtain complete event elements, event relationships, evolution paths and other information from complex network text, and predict the possible relationships between events by using a knowledge graph and event information, so that the evolution law of the events can be better represented and explained.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a method and apparatus for intelligent construction and analysis of event graphs based on the BERT model. Background Technology

[0002] In related technologies, numerous studies both domestically and internationally have explored the analysis of event evolution in online text. These studies capture the interrelationships between events and understand the logic and evolutionary patterns of event evolution by constructing storylines or event knowledge graphs. Storyline-based methods, which mine and analyze online text to form storylines describing event evolution, have limitations such as a lack of detailed event breakdown and description, and an inability to effectively describe the complex relationships between events. Event knowledge graph-based methods, on the other hand, utilize knowledge graphs to represent and store event elements, relationships, and evolutionary path information. Leveraging the data structuring, rich association information, and visualization features of knowledge graphs, they can better represent and explain the evolutionary patterns of events.

[0003] However, when faced with massive amounts of network text with low information density and unstructured complex relationships, the event evolution methods in related technologies have difficulty extracting event information. This leads to increased obstacles in judging the relationships between events through event information, making it difficult to represent the evolutionary laws of events, resulting in poor interpretability and low prediction accuracy, which urgently need to be addressed. Summary of the Invention

[0004] This application provides an intelligent construction and analysis method and apparatus for event graphs based on the BERT model (Bidirectional Encoder Representations from Transformers, deep bidirectional language representation model). This addresses the challenges faced by event evolution methods in related technologies when dealing with massive amounts of network text with complex relationships such as low information density and unstructured nature. These challenges include difficulty in extracting event information, increased obstacles in judging relationships between events, difficulty in representing the evolutionary patterns of events, poor interpretability, and low prediction accuracy.

[0005] The first aspect of this application provides a method for intelligent construction and analysis of event graphs based on the BERT model, comprising the following steps: extracting event information from target network text; converting the extracted events corresponding to the event information into an extracted event graph based on a preset event graph, wherein event fusion is used to merge extracted events reflecting the same real-world events into the same event graph; and based on a pre-trained BERT model, discovering similarities between different events and predicting event relationships through knowledge graphs and event relationship prediction, so as to construct a complete event context and determine the evolution patterns and rules of events.

[0006] Optionally, in one embodiment of this application, the step of extracting event information from the target network text includes: extracting trigger words from the target network text based on a predefined event framework; and determining the event type and argument roles based on the trigger words to obtain the event information.

[0007] Optionally, in one embodiment of this application, the step of converting the extracted events corresponding to the event information into an extracted event graph based on a preset event graph includes: using event fusion to merge extracted events reflecting the same real-world events into a single event graph, which involves: determining the commonalities between different events through the event graph and exploring the relationships between the commonalities between the different events. Specifically, based on the label and name of each argument node, argument nodes with the same name and label are merged, and event nodes connected to the same argument node are determined to have commonalities, so as to infer the relationship through a preset event relationship prediction model.

[0008] Optionally, in one embodiment of this application, the step of converting the extracted events corresponding to the event information into an extracted event graph based on a preset event graph, wherein the extraction of extracted events reflecting the same real-world events into a single event graph using event fusion, further includes: fusing the edges of two events with a coreference relationship; fusing the event nodes of the two events with a coreference relationship, wherein if the two events have the same label and the names of the event nodes of the two events are inconsistent, then the name richer in semantic information is selected as the name of the merged node, and the other name is saved as a candidate name; and merging argument nodes with the same label according to rules that satisfy preset conditions similar to those of the event nodes.

[0009] Optionally, in one embodiment of this application, the model loss function of the BERT model is the cross-entropy loss function, wherein the cross-entropy loss function is:

[0010]

[0011] Where n is the sequence length, b is the number of relations between events, and w a It is the weight of the a-th relation, y ia It is the binary label of the i-th data point on the a-th relation. These are model predictions.

[0012] The second aspect of this application provides an intelligent event graph construction and analysis device based on the BERT model, comprising: an extraction module for extracting event information from target network text; a transformation module for transforming the extracted events corresponding to the event information into an extracted event graph based on a preset event graph, wherein event fusion is used to fuse extracted events reflecting the same real-world events into the same event graph; and a construction module for discovering similarities between different events and predicting event relationships based on a pre-trained BERT model through knowledge graphs and event relationship prediction, so as to construct a complete event context and determine the evolution patterns and rules of events.

[0013] Optionally, in one embodiment of this application, the extraction module includes: an extraction unit, configured to extract trigger words from the target network text based on a predefined event framework; and a determination unit, configured to determine the event type and argument roles based on the trigger words to obtain the event information.

[0014] Optionally, in one embodiment of this application, the transformation module includes: an inference unit, configured to determine commonalities between different events through the event graph and explore the relationships between the commonalities between the different events, wherein, based on the label and name of each argument node, argument nodes with the same name and label are merged, and event nodes connected to the same argument node are determined to have commonalities, so as to infer the relationship through a preset event relationship prediction model.

[0015] Optionally, in one embodiment of this application, the conversion module further includes: a first fusion unit, used to fuse the edges of two events with a coreference relationship; a second fusion unit, used to fuse the event nodes of the two events with a coreference relationship, wherein if the two events have the same label and the names of the event nodes of the two events are inconsistent, the name richer in semantic information is selected as the name of the merged node, and the other name is saved as a candidate name; and an extraction unit, used to extract and merge argument nodes with the same label according to rules that satisfy preset similar conditions to the event nodes.

[0016] Optionally, in one embodiment of this application, the model loss function of the BERT model is the cross-entropy loss function, wherein the cross-entropy loss function is:

[0017]

[0018] Where n is the sequence length, b is the number of relations between events, and w a It is the weight of the a-th relation, y ia It is the binary label of the i-th data point on the a-th relation. These are model predictions.

[0019] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the intelligent construction and analysis method of event graph based on the BERT model as described in the above embodiments.

[0020] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intelligent construction and analysis method for event graphs based on the BERT model.

[0021] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the above-described intelligent construction and analysis method for event graphs based on the BERT model.

[0022] This application's embodiments can extract event information from online text through event extraction, merge the extracted events into a single event graph based on a certain event graph ontology, and finally discover similarities between different events and predict event relationships through knowledge graphs and event relationship prediction, constructing a complete event context and clarifying the evolution patterns and rules of events. This achieves the extraction of event information from complex relationships, the determination of relationships between events, and the prediction of event evolution patterns and rules through knowledge graphs and event relationships. Therefore, it solves the problems of related event evolution methods facing difficulties in extracting event information when dealing with massive amounts of online text with low information density and unstructured complex relationships. This leads to increased obstacles in determining relationships between events based on event information, difficulty in representing event evolution rules, poor interpretability, and low prediction accuracy.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0025] Figure 1 This is a schematic diagram of the overall framework of an event evolution analysis system according to an embodiment of this application;

[0026] Figure 2 This is a flowchart of an intelligent construction and analysis method for event graphs based on the BERT model, according to an embodiment of this application.

[0027] Figure 3 This is a schematic diagram of an event graph ontology according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the structure of the intelligent construction and analysis device for event graphs based on the BERT model according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0030] Figure label:

[0031] 10- Intelligent construction and analysis device for event graph based on BERT model: 100- Extraction module, 200- Transformation module and 300- Construction module; 501- Memory, 502- Processor and 503- Communication interface. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0033] The following describes, with reference to the accompanying drawings, an intelligent event graph construction and analysis method and apparatus based on the BERT model according to embodiments of this application. Addressing the challenges of event evolution methods in the aforementioned background art when dealing with massive amounts of network text with low information density and unstructured complex relationships, which makes event information extraction difficult and hinders the determination of relationships between events, makes it difficult to represent the evolutionary patterns of events, and results in poor interpretability and low prediction accuracy, this application provides an intelligent event graph construction and analysis method based on the BERT model. In this method, event information can be extracted from network text through event extraction. Based on a preset event graph ontology, the extracted events are merged into a single event graph. Finally, through knowledge graphs and event relationship prediction, similarities between different events are discovered, event relationships are predicted, and a complete event context is constructed, thereby clarifying the evolutionary patterns and patterns of events. This achieves the extraction of event information from complex relationships, the determination of relationships between events, and the prediction of event evolution patterns and patterns through knowledge graphs and event relationships. This solves the problems in related technologies where event evolution methods face difficulties in extracting event information when dealing with massive amounts of network text with low information density and unstructured relationships. This leads to increased obstacles in judging the relationships between events through event information, difficulty in representing the evolutionary laws of events, poor interpretability, and low prediction accuracy.

[0034] The following is a schematic diagram of the overall framework of the event evolution analysis system according to an embodiment of this application.

[0035] like Figure 1 As shown, the event evolution analysis system in this application embodiment includes, but is not limited to, parts such as network text, event extraction, event graph representation, and event relationship prediction (event fusion and event context construction).

[0036] Specifically, Figure 2 This is a flowchart illustrating an intelligent construction and analysis method for event graphs based on the BERT model, provided as an embodiment of this application.

[0037] like Figure 2 As shown, this intelligent construction and analysis method for event graphs based on the BERT model includes the following steps:

[0038] In step S201, event information is extracted from the target network text.

[0039] Understandably, online text is one of the main forms of online information, typically existing in various media such as microblogs, instant messaging software, social media, and text messages. Online text often contains a wealth of information about social events, which, with frequent interactions on the internet, spread rapidly, ferment, and have a significant impact on social networks and news media. Therefore, studying and analyzing the events contained in online text can help us understand the current social situation and development trends, and can help predict and address potential future problems.

[0040] In the field of information science, an event refers to a specific action at a particular time and place, involving one or more parties, and can usually be described as a change in state. Events are in a state of development, interdependent, and follow certain evolutionary patterns. Event evolution analysis aims to clarify the various components of an event, the relationships between events, and the evolutionary trajectory of the event, thereby grasping the overall developmental patterns of the event.

[0041] Event extraction here refers to the process of extracting trigger words and arguments from text and determining the event type and argument roles based on predefined event frameworks. Target web text can be understood here as specific text data that needs to be analyzed to identify and extract event information.

[0042] For example, target online text can include, but is not limited to, the following types of text content: social media content, news reports, forum and blog posts, academic articles, and research reports. Event extraction refers to identifying events with specific significance from text. Time information refers to relevant information about the extracted events, such as the type of event, trigger words, participants, time, and location. Extracting event information from target online text involves extracting the original text data containing these events, and then identifying and extracting relevant information about these events from this data.

[0043] Furthermore, embodiments of this application can transform the event extraction task into a multi-label classification task at the lexical level. Each lexical can be labeled as (event type, role), where the role can be a trigger word or an argument role. Since each lexical can be labeled with multiple tags, this means that each lexical can play different roles in different events, thereby solving problems such as event overlap and nesting.

[0044] Optionally, in one embodiment of this application, extracting event information from target web text includes: extracting trigger words from target web text based on a predefined event framework; and determining the event type and argument roles based on the trigger words to obtain event information.

[0045] Based on the descriptions of other embodiments, it is understood that this application can extract time information from target web text. Specifically, it can extract trigger words from target web text using a predefined event framework, and then determine the event type and argument roles based on the trigger words, thereby obtaining event information.

[0046] For example, this application can use a word-level multi-label classification model (MLEE) as an event extraction model to extract event information from target network text. This word-level multi-label classification model relies on the feature representation of a pre-trained language model and can consist of an input layer, a feature representation layer, a probability calculation layer, and an output layer.

[0047] Specifically, at the input layer, text can be lexicalized into a sequence of lexical units consisting of WordPiece embeddings (breaking a word into multiple subwords (WordPieces)), positional embeddings, and segment embeddings. BERT's special lexical units, such as "[CLS]" and "[SEP]", can be embedded at the beginning and end of the sequence. The sequence can then be represented as:

[0048] T = (t1, t2, ..., t) n ),

[0049] Among them, t i For word embedding, n is the sequence length.

[0050] Subsequently, the word sequence can be encoded by BERT in the feature representation layer into a set of semantically rich vectors, which can be represented as follows:

[0051] H = (h1, h2, ..., h) n ),

[0052] Among them, h i It is the word t i The corresponding implicit vector.

[0053] In the probability computation layer, two feedforward neural networks can be used to compute vector a and tensor R, which can be represented as follows:

[0054] a = (a1, a2, ..., a2) n ) = W (a) H+b (a) ,

[0055] R = (r1, r2, ..., r) n ) = W (R) H+b (R) ,

[0056] Among them, a i r represents the probability that the i-th word is an event element. i W is an m-dimensional vector, where m is the number of word types. (a) b (a) W (R) b (R) These are the weights and biases of the two feedforward neural networks, respectively.

[0057] Next, using the elements in a and R, we can calculate the probability that the i-th word is of the k-th type:

[0058]

[0059] Among them, O i,k Let a represent the probability that the i-th lexical in a positive example is of the k-th lexical type. i r is the probability that the i-th element (i.e., the i-th word) in vector a is an event element. i,k It is a vector r i The kth element.

[0060] Finally, the model's output can be represented as a tensor O of shape n×m, where n is the sequence length, m is the number of word types, and O is the number of elements. i,k This represents the probability that the i-th lexical in a positive example is of the k-th lexical type, where each lexical type corresponds to an element role of an event type.

[0061] Furthermore, for multi-label classification with imbalanced categories, embodiments of this application can design a suitable loss function. For example, embodiments of this application can define terms belonging to event elements as positive classes and terms not belonging to event elements as negative classes. For each element role, typically the number of positive classes is small, but the number of negative classes is large.

[0062] Therefore, embodiments of this application can, but are not limited to, use two loss functions for class balancing: circular loss and binary cross-entropy loss. Firstly, in the first stage of model training, embodiments of this application can design a weighted binary cross-entropy loss function, increasing the weight of the positive class and decreasing the weight of the negative class. The loss function can be expressed as follows:

[0063]

[0064] Where n is the sequence length, m is the number of lexical types, and O i,k Y represents the probability that the i-th lexical in a positive example is of the k-th lexical type. i,k Y represents the probability that the i-th lexical in the counterexample is of the k-th lexical type. i,k It is a label with a value of 0 or 1, β i,k and γ i,k The weights can be calculated as follows:

[0065]

[0066] Where I(·) is the indicator function.

[0067] In the second stage, the embodiments of this application can use the following circular loss function for training:

[0068]

[0069] in, and It is a weighting coefficient that is greater than 0. It is the output set of positive examples. It is the output set of negative examples, and exp(·) is the exponential function. and It can be defined as:

[0070]

[0071] Where [·]+ is the cutoff 0 operation, O j,k Y represents the probability that the j-th lexical in a positive example is of the k-th lexical type. j,k This represents the probability that the j-th lexical in the counterexample is of the k-th lexical type.

[0072] Step S202: Based on the preset event graph, the extracted events corresponding to the event information are transformed into an extracted event graph. In this step, event fusion is used to merge extracted events that reflect the same real-world event into the same event graph.

[0073] In some embodiments, the event information that can be extracted from network text is diverse, containing different information about a single event. To obtain a more complete event, it is necessary to integrate this event information to a certain extent.

[0074] Therefore, in this embodiment of the application, after extracting event information from network text, the events corresponding to the extracted event information can be transformed into an extracted event graph based on a certain event graph, and then the extracted events that reflect the same event can be merged into the same event graph.

[0075] The process will now be explained in more detail.

[0076] Optionally, in one embodiment of this application, based on a preset event graph, the extracted events corresponding to the event information are transformed into an extracted event graph. The extracted events reflecting the same real-world events are merged into a single event graph using event fusion, which includes: determining the commonalities between different events through the event graph and exploring the relationships between the commonalities between different events. Specifically, based on the label and name of each argument node, argument nodes with the same name and label are merged, and event nodes connected to the same argument node are determined to have commonalities, so as to infer the relationship through a preset event relationship prediction model.

[0077] In actual execution, different events share certain commonalities and differences. To find the connections between events, it is necessary to identify the relationships between these commonalities. Therefore, in this embodiment of the application, when transforming the extracted events corresponding to event information into an extracted event graph, the commonalities between different events can be determined first through the event graph. Then, the relationships between these commonalities can be explored, facilitating the subsequent inference of relationships between events using a certain time relationship prediction model.

[0078] For example, Figure 4 This is a schematic diagram of an event graph ontology according to an embodiment of this application. Figure 4 As shown, the event graph constructed in this embodiment includes, but is not limited to, two types of nodes: event nodes and argument nodes. An event node represents an event, with each event node labeled by its event type and its trigger word serving as the node name. An argument node represents an argument, with the argument serving as the node name and its argument type serving as the node label.

[0079] Furthermore, the event graph in this embodiment also includes, but is not limited to, two types of edges: event relationship edges and argument role edges. Event relationship edges connect two event nodes, representing the relationship between the two events. Argument role edges connect an event node and an argument node, representing the role of the argument in the event. Setting events and arguments as nodes separately avoids modeling the nodes in the graph as a composite structure, facilitating subsequent analysis and processing of the graph.

[0080] By using event graphs, embodiments of this application can identify commonalities between different events and explore their relationships. For example, by examining the label and name of each argument node, argument nodes with the same name and label are merged, and event nodes connected to the same argument node are considered to have commonalities. The relationships between them can then be inferred through subsequent event relationship prediction models.

[0081] Optionally, in one embodiment of this application, based on a preset event graph, the extracted events corresponding to the event information are transformed into an extracted event graph. The process of merging extracted events reflecting the same real-world events into a single event graph using event fusion further includes: merging the edges of two events with a coreference relationship; merging the event nodes of two events with a coreference relationship, wherein if two events have the same label and the names of their event nodes are inconsistent, the name richer in semantic information is selected as the name of the merged node, and the other name is saved as a candidate name; and merging argument nodes with the same label according to rules that satisfy preset conditions similar to those of the event nodes.

[0082] In other embodiments, this application considers not only the relationships between events but also the possibility of coreference between them. Coreference between events refers to the phenomenon that two or more events conceptually point to the same underlying event. This relationship is particularly important in fields such as text processing, information extraction, and knowledge graph construction, as it helps to better understand event information in text, as well as the connections and evolution between events.

[0083] For example, coreference relationships can be categorized into the following key points: **Same Concept:** If two event references are conceptually related to the same underlying event, then they have a coreference relationship. For instance, if the specific time, location, participants, etc., of two events are not entirely identical, but they essentially describe the same event, they can be considered coreferences. **Different Parameters:** Coreference relationships allow for differences in the parameters of event references. For example, when describing the same attack, different numbers of deaths or injuries might be mentioned, but as long as these references all point to the same attack, they still constitute a coreference relationship. **Classification and Recognition:** Recognizing coreference relationships is an important task in natural language processing. It requires the system to accurately understand event information in text and determine which event references point to the same underlying event. **Relationship with Event Sequence Relationships:** While events have an order relationship, text event references also have coreference relationships. For example, event sequence tasks study how to group and sort events in text documents belonging to the same script, while coreference relationships, based on these groupings and sortings, further determine which event references point to the same underlying event.

[0084] Furthermore, embodiments of this application can, but are not limited to, use a rule-based method to fuse two event graphs that have a coreference relationship. For example, firstly, the edges of two events with a coreference relationship are fused. The edges connected to event nodes can be determined by a predefined event framework; therefore, specific types of events have defined edges, and these edges can be fused based on whether their names are the same.

[0085] Next, embodiments of this application can merge two event nodes. For example, if two event nodes have the same label for their events, and their names are inconsistent, a name richer in semantic information can be selected as the name of the merged node, and the other name can be saved as a candidate name. Finally, argument nodes with the same label are merged using rules similar to those for event nodes.

[0086] The embodiments of this application can help explore the connections between the constituent elements of different events by examining the relationships between different nodes, and at the same time, it is beneficial to discover the commonalities between different events and the evolutionary laws of events.

[0087] Step S203: Based on the pre-trained BERT model, the similarities between different events are discovered and the relationships between events are predicted through knowledge graph and event relationship prediction, so as to construct a complete event context and determine the evolution pattern and laws of events.

[0088] Optionally, in one embodiment of this application, the model loss function of the BERT model can be the cross-entropy loss function, wherein the cross-entropy loss function can be expressed as:

[0089]

[0090] Where n is the sequence length, b is the number of relations between events, and w a It is the weight of the a-th relation, y ia It is the binary label of the i-th data point on the a-th relation. These are model predictions.

[0091] Those skilled in the art will understand that the BERT model (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on Transformers. Trained on large-scale unlabeled corpora, it obtains representations of text containing rich semantic information—that is, semantic representations of the text. These semantic representations are then fine-tuned for specific NLP (Natural Language Processing) tasks and ultimately applied to those tasks. The core feature of the BERT model lies in its training methods using Masked Language Model (MLM) and Next Sentence Prediction (NSP), enabling the model to learn bidirectional contextual information within the text, thereby generating deep bidirectional language representations.

[0092] The training process of the BERT model consists of two main steps: pre-training and fine-tuning. During pre-training, the BERT model learns word-level semantic information through the MLM task and sentence-level semantic information through the NSP task. The fine-tuning stage involves fine-tuning the pre-trained BERT model according to specific tasks to adapt it to different downstream tasks, such as text classification and question answering.

[0093] The BERT model has achieved great success in the field of natural language processing and is widely used in various NLP tasks, such as sentiment classification, intent recognition, and question matching. Furthermore, the BERT model supports training with custom data; by providing custom corpora and labels, the model can be trained in a customized manner to adapt to specific application scenarios.

[0094] As one possible approach, after merging extracted events reflecting the same real-world events into a single event graph, embodiments of this application can also utilize a pre-trained BERT model to discover similarities between different events and predict event relationships through knowledge graphs and event relationship prediction, thereby constructing a complete event context and determining the evolution patterns and laws of events.

[0095] The event relationship prediction in this application embodiment can be, but is not limited to, predicting coreference relationships, causal relationships, and temporal relationships between events. The event relationship prediction task can be viewed as a multi-classification task, where each output type of the model corresponds to one event relationship.

[0096] For example, this application can design a relation prediction model based on a pre-trained language model, consisting of a serialization layer, an encoding layer, and a relation prediction layer.

[0097] In the serialization layer, since pre-trained language models like XLNet can only accept sequences as input, structured event information is first serialized into a sequence of tokens. Therefore, a text containing event information can be represented as:

[0098] T = (t1, t2, ..., t) n ),

[0099] Where n is the sequence length.

[0100] Next, this embodiment of the application can assume that the text contains K events, and represent the roles of these elements as special terms added to the XLNet vocabulary. For example, if the role of an element is "Trigger", then "[TRIGGER]" and "[ / TRIGGER]" are added to the vocabulary. Subsequently, these special terms can be used to annotate the event elements in the text, and the annotation can be represented as follows:

[0101] seq1 = (t1, ..., [R] k ], t k,1 , t k,2 , ..., t k,m ,[ / R k ],……,t n ),

[0102] Among them, [R k ],[ / R k [SEP] refers to the special token defined above. Finally, the sequence pairs seq1 and seq2 corresponding to the two events are concatenated, and the token "[SEP]" is placed between the two sequences. This yields the following representation:

[0103] seq=([CLS],seq1,[SEP],seq2,[SEP])

[0104] CLS and SEP are special lexical units in BERT, and seq represents a sequence pair generated by concatenating sequences seq1 and seq2.

[0105] At the encoding layer, embodiments of this application can use XLNet as an encoder, which can encode argument roles, event elements, and contextual information into latent vectors.

[0106] At the relation prediction layer, embodiments of this application can use the latent vector h corresponding to the lexical units “[CLS]”, “[TRIGGER]”, and “[ / TRIGGER]”. cls hth h te Concatenate and perform full connectivity, that is:

[0107] o = softmax(W × [h] cls h th h te ]+b)

[0108] Where W and b are the weights and biases of the feedforward neural network, o is the output vector representing the probabilities of different relationships, and softmax(·) is the activation function.

[0109] Furthermore, to reduce losses in model applications, embodiments of this application may also define a model loss function, such as the cross-entropy loss function, which can be expressed as:

[0110]

[0111] Where n is the sequence length, b is the number of relations between events, and w a It is the weight of the a-th relation, y ia It is the binary label of the i-th data point on the a-th relation. These are model predictions.

[0112] The event graph intelligent construction and analysis method based on the BERT model proposed in this application can extract event information from network text through event extraction, merge the extracted events into a single event graph according to a preset event graph ontology, and finally discover the similarities between different events and predict event relationships through knowledge graph and event relationship prediction, thus constructing a complete event context and clarifying the evolution patterns and rules of events. This achieves the extraction of event information from complex relationships, the determination of relationships between events, and the prediction of event evolution patterns and rules through knowledge graph and event relationships. Therefore, it solves the problems of related event evolution methods facing massive, low-information-density, unstructured, and complex network texts, which make event information extraction difficult, leading to increased obstacles in determining relationships between events, difficulty in representing event evolution rules, poor interpretability, and low prediction accuracy.

[0113] Next, referring to the accompanying drawings, we describe the intelligent construction and analysis device for event graphs based on the BERT model proposed in the embodiments of this application.

[0114] Figure 5 This is a schematic diagram of the structure of the intelligent construction and analysis device for event graphs based on the BERT model according to an embodiment of this application.

[0115] like Figure 5As shown, the BERT-based intelligent event graph construction and analysis device 10 includes: an extraction module 100, a transformation module 200, and a construction module 300.

[0116] The extraction module 100 is used to extract event information from the target network text.

[0117] The transformation module 200 is used to transform the extracted events corresponding to the event information into an extracted event graph based on a preset event graph. In this process, event fusion is used to merge extracted events that reflect the same real-world event into the same event graph.

[0118] Module 300 is used to build a complete event context and determine the evolution patterns and laws of events based on a pre-trained BERT model by using knowledge graphs and event relationship prediction.

[0119] Optionally, in one embodiment of this application, the extraction module 100 includes an extraction unit and a determination unit.

[0120] The extraction unit is used to extract trigger words from the target network text based on a predefined event framework.

[0121] The determination unit is used to determine the event type and argument roles based on the trigger word in order to obtain event information.

[0122] Optionally, in one embodiment of this application, the conversion module 200 includes: an inference unit.

[0123] The inference unit is used to determine the commonalities between different events through the event graph and explore the relationships between the commonalities between different events. Specifically, based on the label and name of each argument node, argument nodes with the same name and label are merged, and event nodes connected to the same argument node are determined to have commonalities, so as to infer the relationship through a preset event relationship prediction model.

[0124] Optionally, in one embodiment of this application, the conversion module 200 further includes: a first fusion unit, a second fusion unit, and an acquisition unit.

[0125] The first fusion unit is used to fuse the edges of two events that have a common reference relationship.

[0126] The second fusion unit is used to fuse the event nodes of two events with a common reference relationship. If the two events have the same label and the names of the event nodes of the two events are different, the name with richer semantic information is selected as the name of the merged node, and the other name is saved as a candidate name.

[0127] The take unit is used to take argument nodes with the same label that meet the preset conditions of the event node and are merged by the rules.

[0128] Optionally, in one embodiment of this application, the model loss function of the BERT model can be the cross-entropy loss function, wherein the cross-entropy loss function can be expressed as:

[0129]

[0130] Where n is the sequence length, b is the number of relations between events, and w a It is the weight of the a-th relation, y ia It is the binary label of the i-th data point on the a-th relation. These are model predictions.

[0131] It should be noted that the foregoing explanation of the embodiment of the intelligent construction and analysis method of event graph based on the BERT model also applies to the intelligent construction and analysis device of event graph based on the BERT model in this embodiment, and will not be repeated here.

[0132] The BERT-based intelligent event graph construction and analysis device proposed in this application can extract event information from network text through event extraction, merge the extracted events into a single event graph according to a preset event graph ontology, and finally discover the similarities between different events and predict event relationships through knowledge graph and event relationship prediction, thus constructing a complete event context and clarifying the evolution patterns and rules of events. This achieves the extraction of event information from complex relationships, the determination of relationships between events, and the prediction of event evolution patterns and rules through knowledge graph and event relationships. Therefore, it solves the problems of related event evolution methods facing massive, low-information-density, unstructured, and complex network texts, which make event information extraction difficult, leading to increased obstacles in determining relationships between events, difficulty in representing event evolution rules, poor interpretability, and low prediction accuracy.

[0133] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0134] The memory 501, the processor 502, and the computer program stored on the memory 501 and capable of running on the processor 502.

[0135] When the processor 502 executes the program, it implements the intelligent construction and analysis method of event graph based on the BERT model provided in the above embodiments.

[0136] Furthermore, electronic devices also include:

[0137] Communication interface 503 is used for communication between memory 501 and processor 502.

[0138] The memory 501 is used to store computer programs that can run on the processor 502.

[0139] The memory 501 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0140] If the memory 501, processor 502, and communication interface 503 are implemented independently, then the communication interface 503, memory 501, and processor 502 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0141] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0142] Processor 502 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0143] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described intelligent construction and analysis method for event graphs based on the BERT model.

[0144] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the intelligent construction and analysis method of event graph based on the BERT model provided in this application.

[0145] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0146] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0147] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0148] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0149] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0150] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0151] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0152] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for intelligent construction and analysis of event graphs based on the BERT model, characterized in that, Includes the following steps: Extract event information from target network text; Based on a preset event graph, the extracted events corresponding to the event information are transformed into an extracted event graph. Event fusion is used to merge extracted events reflecting the same real-world event into the same event graph. Based on the pre-trained BERT model, the similarities between different events are discovered and the relationships between events are predicted through knowledge graphs and event relationship prediction, so as to construct a complete event context and determine the evolution pattern and laws of events. The process of transforming extracted events corresponding to the event information into an extracted event graph based on a preset event graph includes: using event fusion to merge extracted events reflecting the same real-world events into a single event graph; determining commonalities between different events through the event graph and exploring the relationships between these commonalities; and merging argument nodes with the same name and label based on the label and name of each argument node, and determining that event nodes connected to the same argument node have commonalities, so as to infer the relationships through a preset event relationship prediction model. The process involves transforming extracted events corresponding to the event information into an extracted event graph based on a preset event graph. Event fusion is used to merge extracted events reflecting the same real-world event into a single event graph. This includes: merging edges of two events with a coreference relationship; merging event nodes of the two events with a coreference relationship, wherein if the two events have the same label and the names of their event nodes are inconsistent, the name richer in semantic information is selected as the name of the merged node, and the other name is saved as a candidate name; and merging argument nodes with the same label according to rules that satisfy preset conditions similar to those of the event nodes.

2. The method according to claim 1, characterized in that, The extraction of event information from the target network text includes: Based on a predefined event framework, trigger words are extracted from the target network text; The process of determining the event type and argument roles based on the trigger words is used to obtain the event information.

3. The method according to claim 1, characterized in that, The model loss function of the BERT model is the cross-entropy loss function, wherein the cross-entropy loss function is: ， in, n It is the sequence length. b It is the relation number of the event. It is the first a The weight of the relationship, It is the first i The data in the first a Binary labels on a relation, These are model predictions.

4. A device for intelligent construction and analysis of event graphs based on the BERT model, characterized in that, The method for intelligent construction and analysis of event graphs based on the BERT model as described in any one of claims 1-3 is adopted, wherein the apparatus comprises: The extraction module is used to extract event information from target network text; The conversion module is used to convert the extracted events corresponding to the event information into an extracted event graph based on a preset event graph. In this process, event fusion is used to merge extracted events that reflect the same real-world event into the same event graph. The building module is used to discover similarities between different events and predict event relationships based on the pre-trained BERT model through knowledge graphs and event relationship prediction, so as to build a complete event context and determine the evolution pattern and laws of events.

5. The apparatus according to claim 4, characterized in that, The extraction module includes: An extraction unit is used to extract trigger words from the target network text based on a predefined event framework; The determining unit is used to determine the event type and argument role based on the trigger word in order to obtain the event information.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the intelligent construction and analysis method of the event graph based on the BERT model as described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the intelligent construction and analysis method of the event graph based on the BERT model as described in any one of claims 1-3.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, it is used to implement the intelligent construction and analysis method of event graph based on the BERT model as described in any one of claims 1-3.

Images