Event causal relationship extraction method and device based on contextual time relationship

Abstract

The present invention discloses a method and device for extracting event causal relationships based on contextual temporal relationships, including: annotating Chinese text data of urban underground space disasters with causal relationships, inputting the Chinese text data of disasters with causal relationships annotated into a BERT‑BiLSTM‑CRF model, and extracting causal relationships of urban underground space disaster events; when the BERT layer in the BERT‑BiLSTM‑CRF model outputs a word vector, concatenating the encoding vector of the time entity with the word vector output by the BERT layer to obtain a word vector containing the time relationship, and then inputting the word vector containing the time relationship into the BiLSTM layer for processing. The present invention proposes a causal relationship extraction method of a BERT‑BiLSTM‑CRF model that integrates a temporal feature matrix, which improves the accuracy of causal relationship recognition of urban underground space disaster events, and improves the accuracy and reliability of causal relationship extraction.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a method and device for extracting event causal relationships based on contextual time relationships. Background Art

[0002] As modern cities continue to explore underground space, the importance of urban underground space in urban economy and transportation has become increasingly prominent. Urban underground space disasters are the main factor restricting its development and utilization. Therefore, the prevention and control of underground space disasters is of vital importance. Scientific, efficient, economical and timely disaster prevention and control is a strategic requirement for national disaster reduction and sustainable urban development.

[0003] A knowledge graph is a graphical data structure used to represent the relationships and attributes between entities (such as people, places, concepts, etc.). It abstracts the complex relationships in the real world into a graph so that machines can understand and process these relationships, thereby achieving semantic knowledge representation and reasoning.

[0004] The key technology of knowledge graph is knowledge extraction, which refers to identifying, formatting, and filtering knowledge points in text data, and storing these knowledge points in a specific form in the knowledge database. It covers two key steps: entity extraction and relationship extraction, aiming to improve the availability and reusability of information. The mainstream methods of entity extraction mainly include rule-based and dictionary-based methods, supervised learning-based methods, and deep learning-based methods; while relationship extraction determines the semantic relationship contained in the text through the context of the context, and causal relationship extraction is a more difficult task in relationship extraction.

[0005] Due to the different characteristics, causes and specific forms of disasters in different regions, these uncertainties make it difficult for pre-made emergency plans to effectively respond to disasters and fully protect people's lives and property. Therefore, more flexible and targeted disaster prevention and control strategies are needed. Therefore, based on informatization and using knowledge graphs as a means, it is very important to summarize the knowledge of typical urban underground space disasters into a database, provide people with a channel to obtain knowledge of typical urban underground space disasters, and establish a platform for displaying and searching urban underground space disaster knowledge based on knowledge graphs. Summary of the invention

[0006] An object of embodiments of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages to be described below.

[0007] The embodiments of the present invention provide a method, device, electronic device and storage medium for extracting event causal relationships based on contextual temporal relationships. To this end, the present invention provides the following technical solutions:

[0008] In a first aspect, a method for extracting event causal relationships based on contextual temporal relationships is provided, comprising:

[0009] Entity extraction is performed on Chinese text data of urban underground space disasters to obtain time entities;

[0010] Annotating the cause-effect relationship of the Chinese text data of the urban underground space disaster;

[0011] The Chinese text data of disasters with causal relationship annotations were input into the BERT-BiLSTM-CRF model to extract the causal relationship of urban underground space disaster events.

[0012] Among them, when the BERT layer in the BERT-BiLSTM-CRF model outputs a word vector, the encoding vector of the time entity is concatenated with the word vector output by the BERT layer to obtain a word vector containing a time relationship, and then the word vector containing the time relationship is input into the BiLSTM layer for processing.

[0013] Preferably, the event causal relationship extraction method based on contextual time relationship further includes:

[0014] Before performing entity extraction on the Chinese text data of urban underground space disasters, the Chinese text data of urban underground space disasters is screened and pre-processed for noise reduction, the Chinese text data of disasters used for entity extraction is retained, and then word segmentation and labeling operations are performed. During the labeling operation, all time-related information is labeled as an entity type to obtain a time entity labeling data set;

[0015] When the Chinese text data of the urban underground space disaster is annotated with a causal relationship, the time entity annotation data set is superimposed to obtain the disaster text data with the causal relationship annotation of the time entity added;

[0016] The disaster text data with the causal relationship annotations of the time entity is input into the BERT-BiLSTM-CRF model to complete the extraction of the causal relationship of urban underground space disaster events.

[0017] Preferably, in the event causal relationship extraction method based on contextual time relationship, the method of annotating the causal relationship of the urban underground space disaster Chinese text data comprises:

[0018] First, the Chinese text data of urban underground space disasters are screened and pre-processed for noise reduction, and the Chinese text data of disasters used for relationship extraction is retained. Then, the word segmentation operation is performed, and then the semi-supervised learning method is used to annotate the causal relationship. The manually annotated text data set is input into the LTP tool for learning, and then the LTP tool annotates the remaining data sets.

[0019] The causal relationship annotation adopts the BIO annotation rule to annotate the trigger words representing the causal relationship, and annotate the causal entity introduced by each trigger word as "BX", "IX" or "O", where "BC" indicates that the position of the character is the starting part of the cause fragment, "IC" indicates that the position of the character is the non-starting position of the cause fragment, "BR" indicates that the position of the character is the starting part of the result fragment, "IR" indicates that the position of the character is the non-starting position of the result fragment, and "O" indicates that the element does not belong to any of the above types.

[0020] Preferably, in the event causal relationship extraction method based on contextual time relationship, the word vector containing the time relationship enters the BiLSTM layer again, and the BiLSTM layer performs forward and reverse information processing on it using the time relationship, and then uses the CRF component to model the label of the text sequence output by the BiLSTM component.

[0021] Preferably, in the event causal relationship extraction method based on contextual time relationship, when the disaster Chinese text data used for entity extraction is annotated, the BIO part-of-speech tagging method is adopted, wherein "B" represents the start node of the entity, and "I" represents all sequence labels after the start node. The LTP tool is used for part-of-speech tagging, and the entity type is abstracted into five categories, namely "LOC", "PER", all time-related information, "ORG" and "DIS"; (1) LOC: including underground space locations such as underground passages, subways, underground shopping malls, underground parking lots and other location information; (2) PER: including information such as names and casualties; (3) T: all time-related information; (4) ORG: including rescue agencies and other social organizations; (5) DIS: including information such as the type of underground space disasters and the losses caused by disasters.

[0022] The disaster Chinese text data processed as above is input into the ALBERT-BiLSTM-CRF model to complete entity extraction, and the time entity is also obtained at this time.

[0023] Preferably, in the event causal relationship extraction method based on contextual time relationship, the urban underground space disaster Chinese text data selects the urban underground space flood disaster and pipeline accident disaster event text data.

[0024] In a second aspect, a device for extracting event causal relationships based on contextual temporal relationships is provided, comprising:

[0025] Relationship extraction preprocessing module, which is used to annotate the causal relationship of Chinese text data on urban underground space disasters;

[0026] The BERT-BiLSTM-CRF model is used to extract event causal relationships from input Chinese text data of disasters that have been annotated with causal relationships. When the BERT layer in the BERT-BiLSTM-CRF model outputs a word vector, the encoding vector of the time entity of the Chinese text data of urban underground space disasters is concatenated with the word vector output by the BERT layer to obtain a word vector containing a time relationship, and then the word vector containing the time relationship is input into the BiLSTM layer.

[0027] Preferably, the event causal relationship extraction device based on contextual time relationship further includes:

[0028] An entity extraction preprocessing module is used to label the Chinese text data of urban underground space disasters according to entity type classification, where one entity type is all time-related information, and obtain a time entity labeling data set;

[0029] When the relationship extraction preprocessing module performs causal relationship annotation on the urban underground space disaster Chinese text data, the time entity annotation data set is superimposed to obtain disaster text data with causal relationship annotation of the time entity added;

[0030] The BERT-BiLSTM-CRF model is used to extract event causal relationships from the input disaster text data with causal relationship annotations of time entities.

[0031] Optionally, in the event causal relationship extraction device based on contextual time relationship, the method for the relationship extraction preprocessing module to perform causal relationship annotation on the urban underground space disaster Chinese text data includes:

[0032] First, the Chinese text data of urban underground space disasters are screened and pre-processed for noise reduction, and the Chinese text data of disasters used for relationship extraction is retained. Then, the word segmentation operation is performed, and then the semi-supervised learning method is used to annotate the causal relationship. The manually annotated text data set is input into the LTP tool for learning, and then the LTP tool annotates the remaining data sets.

[0033] The causal relationship annotation adopts the BIO annotation rule to annotate the trigger words representing the causal relationship, and annotate the causal entity introduced by each trigger word as "BX", "IX" or "O", where "BC" indicates that the position of the character is the starting part of the cause fragment, "IC" indicates that the position of the character is the non-starting position of the cause fragment, "BR" indicates that the position of the character is the starting part of the result fragment, "IR" indicates that the position of the character is the non-starting position of the result fragment, and "O" indicates that the element does not belong to any of the above types.

[0034] Optionally, in the event causal relationship extraction device based on contextual temporal relationship, in the BERT-BiLSTM-CRF model, the word vector containing the temporal relationship enters the BiLSTM layer again, and the BiLSTM layer performs forward and reverse information processing on it using the temporal relationship, and then uses the CRF component to perform label modeling on the text sequence output by the BiLSTM component.

[0035] Optionally, in the event causal relationship extraction device based on contextual time relationship, when the relationship extraction preprocessing module performs an annotation operation on the disaster Chinese text data used for entity extraction, the BIO part-of-speech tagging method is adopted, wherein "B" represents the start node of the entity, and "I" represents all sequence labels after the start node. The LTP tool is used for part-of-speech tagging, and the entity type is abstracted into five categories, namely "LOC", "PER", all time-related information, "ORG" and "DIS"; (1) LOC: including underground space locations such as underground passages, subways, underground shopping malls, underground parking lots and other location information; (2) PER: including information such as names and casualties; (3) T: all time-related information; (4) ORG: including rescue agencies and other social organizations; (5) DIS: including information such as the type of underground space disasters and the losses caused by the disasters.

[0036] Optionally, the event causal relationship extraction device based on contextual time relationship further includes:

[0037] The ALBERT-BiLSTM-CRF model is used to extract entities from the disaster Chinese text data processed by the input relation extraction preprocessing module.

[0038] Optionally, the urban underground space disaster Chinese text data selects text data of flood disasters and pipeline disaster events in urban underground spaces.

[0039] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes any one of the methods described.

[0040] According to a fourth aspect, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, any of the methods described above is implemented.

[0041] The embodiments of the present invention have at least the following beneficial effects:

[0042] In the causal relationship extraction of urban underground space disaster events, the present invention splices the encoding vector of the time entity in the text data with the word vector output by BERT, and adds the word vector of the time entity to the original word vector output by BERT. By splicing or adding the encoding vector of the time entity with the word vector of BERT, the model's perception of the temporal context can be enhanced, thereby better capturing the causal relationship between events. At the same time, adding time entities to manually annotated data can help the model learn the association between time entities and causal relationships. When annotating data, time entities and causal relationships can be annotated at the sentence level or the fragment level so that the model learns the importance of time entities to causal relationship extraction and improves the accuracy and reliability of causal relationship extraction.

[0043] The present invention proposes a causal relationship extraction method of the BERT-BiLSTM-CRF model integrating the time feature matrix. The model adds time features on the basis of the BERT-BiLSTM-CRF neural network, thereby improving the accuracy of causal relationship identification of urban underground space disaster events. The average precision, recall rate and F1 value are 89.46%, 88.32% and 88.89% respectively, indicating that the model has certain advantages in extracting the causal relationship of urban underground space disaster events.

[0044] Other advantages, objectives and features of the embodiments of the present invention will be reflected in part through the following description, and in part will be understood by those skilled in the art through study and practice of the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is an example of original text data of urban underground space disasters in one embodiment of the present invention.

[0046] Figure 2 This is an example of text data after word segmentation of urban underground space disasters in one embodiment of the present invention.

[0047] Figure 3 This is an example of manual annotation in entity extraction in one of the embodiments of the present invention.

[0048] Figure 4 This is a diagram of the ALBERT-BiLSTM-CRF model architecture in one of the embodiments of the present invention.

[0049] Figure 5 This is a flowchart of the operation of the ALBERT-BiLSTM-CRF model in one of the embodiments of the present invention.

[0050] Figure 6 This is an example of part-of-speech tagging rules in relation extraction in one of the embodiments of the present invention.

[0051] Figure 7 This is an example of manual annotation in relation extraction in one of the embodiments of the present invention.

[0052] Figure 8 This is a diagram of the architecture of a BERT-BiLSTM-CRF causal relationship extraction model that integrates temporal feature vectors in one of the embodiments of the present invention.

[0053] Fig. 9 This is a flowchart of the operation of the BERT-BiLSTM-CRF causal relationship extraction model integrating the time feature vector in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the embodiments according to the description.

[0055] In the process of constructing the knowledge graph of urban underground space disasters, knowledge extraction is the most basic step, and the results of knowledge extraction often determine the overall quality of the knowledge graph. In this application, according to the characteristics of underground space disaster knowledge data, the ALBERT-BiLSTM-CRF named entity recognition model and the event causal relationship extraction method based on contextual time relationship are proposed. These two methods have shown better results in the recognition and extraction of knowledge in the field of urban underground space disasters.

[0056] 1.1ALBERT-BiLSTM-CRF Named Entity Recognition Model

[0057] Named Entity Recognition (NER) is the most basic and critical step in building a knowledge graph. Its task is to identify entities with specific meanings in the text (such as names of people, places, and institutions) and to label these entities for the subsequent construction of a knowledge graph. The named entity recognition task automatically extracts and analyzes a large amount of text data to discover important information and relationships, which is conducive to the automatic processing and mining of information. Named entities related to urban underground space disasters are identified from the text. These named entities include the following types:

[0058] (1) Underground space: including urban underground transportation facilities, underground commercial facilities, and underground entertainment facilities;

[0059] (2) Disaster type: including fire, gas leakage, flood and earthquake;

[0060] (3) Loss: including casualties and property losses;

[0061] (4) Rescue measures: including rescue team, rescue equipment and rescue time, etc.

[0062] 1.1.1 Data Preprocessing

[0063] The urban underground space disaster text data was manually screened, and the text data was preprocessed for text noise reduction using Python scripts, and finally about 1 million words of Chinese text data were retained. Figure 1 and Figure 2 As shown in the figure, the natural language processing tool (LTP) of Harbin Institute of Technology is used for word segmentation operation, and the collected text data of urban underground space flood disasters and pipeline accident disasters are analyzed. Through manual interpretation, the entities related to urban underground space, disaster type, loss situation and rescue measures are manually labeled, and the results are created as a dictionary and added to the LTP word segmentation tool. Finally, the Chinese words contained in a large amount of Chinese text data are separated by spaces, which facilitates the input of text data into the neural network in subsequent knowledge extraction operations.

[0064] Part-of-speech tagging is the process of tagging the part of speech of each word in a text. In the task of named entity recognition, part-of-speech tagging is used as an auxiliary tool to identify the type and contextual relationship of an entity. This paper adopts the "BIO" part-of-speech tagging method, where "B" represents the start node of the entity and "I" represents all sequence labels after the start node. The LTP tool is used for part-of-speech tagging. To improve the accuracy, the entity types are abstracted into five categories, namely (1) LOC: including underground space locations such as underground passages, subways, underground shopping malls, underground parking lots, and other location information; (2) PER: including information such as names and casualties; (3) T: all time-related information; (4) ORG: including rescue organizations and other social organizations; (5) DIS: including information such as the type of underground space disasters and the losses caused by disasters. Figure 3 This is a manually labeled example. 1.1.2 ALBERT-BiLSTM-CRF model

[0065] This paper takes the Chinese text data of urban underground space disaster events as the research object and constructs the ALBERT+BiLSTM+CRF model to realize the named entity recognition research of typical urban underground space disaster knowledge. Figure 4 As shown in the figure, the model consists of ALBERT layer, BiLSTM layer and CRF layer from bottom to top. Figure 5To run the model, ALBERT is used as a feature extractor to convert the input urban underground space disaster text data into a high-dimensional vector representation to extract semantic information, which can help the model better understand the contextual relationship in the text; then, the bidirectional LSTM (BiLSTM) layer is used to model the contextual information of the text data to capture the dependencies in the context; finally, the conditional random field (CRF) layer is used to optimize the output tag sequence of the model as a whole to make the output tag sequence of the model more accurate.

[0066] 1.1.3 Experimental environment and evaluation indicators

[0067] This experiment uses the Tensorflow1.14 framework and python3.7 environment. The GPU model is NvidiaRTX 3060 and the video memory is 12GB. The pre-trained model uses the trained Chinese ALBERT model, which has 12MB of parameters, 12 layers, and a size of 40MB.

[0068] The relevant parameters of the named entity recognition task model are as follows: After preprocessing the Chinese text data, a total of 18,109 sentences were obtained, and other model parameter configurations are shown in Table 1.

[0069] Table 1 Parameter configuration

[0070]

[0071] The ALBERT-BiLSTM-CRF model uses precision, recall and F1 value as the main criteria for evaluating model performance during training. The specific result classification is detailed in Table 2. Precision measures the proportion of samples predicted by the model as positive examples that are actually positive; recall measures the proportion of samples that are actually positive examples that are correctly predicted by the model as positive; F1 value is the harmonic mean of precision and recall, which provides a single metric to evaluate the performance of the model. Its expression is as follows:

[0072]

[0073]

[0074]

[0075] Table 2 Classification results

[0076]

[0077] 1.1.4 Experimental results and evaluation

[0078] Due to the large workload of manual labeling, an entity labeling dataset that is a 1:1 fusion of a manually labeled dataset and a similar People’s Daily dataset was used for experiments. The final dataset size was 3.1MB, which was divided into training set, validation set, and test set in a ratio of 8:1:1 for experiments.

[0079] Table 3 shows the experimental results of different types of entities. From this table, we can see that the F1 value of the model is relatively high in the location entity, population entity and disaster entity part, which can reach 98.14%. This is mainly because the entity nouns of location, population and disaster types are relatively fixed and have obvious boundaries. The effect is poor in the organization entity and time entity part. After analysis, it is found that the reasons may be as follows: (1) The types and names of organizations are updated frequently, and the manually labeled data set is limited; (2) When labeling time entities, time relative nouns such as "that day" and "later" are taken into account, which have no clear boundaries; (3) The number of time entity labels and their quantity is less than that of the other four, which affects the entity recognition results to a certain extent.

[0080] Table 3 Named entity recognition results

[0081]

[0082] In order to verify the advantages of the ALBERT-BiLSTM-CRF model, three mainstream named entity recognition models were selected for comparative experiments, namely the BiLSTM-CRF model, the BERT-CRF model, and the BERT-BiLSTM-CRF model.

[0083] This paper selected the Boson dataset for comparative experiments to verify the applicability of the model. The entity types of the Boson dataset are divided into six types: company_name, Location, org_name, person_name, product_name, and time. They are similar to the types of underground space disaster entities. The parameters of the four models are set to be the same, with Batch_Size of 16, epoch value of 15, and lr value of 0.00001. The experimental results are shown in Table 4. It can be clearly seen that the ALBERT-BiLSTM-CRF model has the best performance, followed by the BERT-BiLSTM-CRF model, and the BiLSTM-CRF model has the worst performance.

[0084] Table 4 Experimental results of Boson dataset

[0085]

[0086] The urban underground space dataset was input into four models for experiments, ensuring the same parameters. The results are shown in Table 5. The performance of the four models is improved compared with the Boson data, mainly because the Boson dataset has more entity types and there are entity types such as company names and product names that are difficult to identify. In the recognition task of the urban underground space disaster dataset, the ALBERT-BiLSTM-CRF model has the best performance and is significantly improved compared with the BERT-BiLSTM-CRF model. This is because the ALBERT model improves the BERT model through a series of technical means, making it more efficient and better. It can be shown that the ALBERT-BiLSTM-CRF model is more suitable for the urban underground space disaster named entity recognition task.

[0087] Table 5 Experimental results of urban underground space disaster dataset

[0088]

[0089] In summary, the ALBERT-BiLSTM-CRF proposed in this paper is a deep learning model for named entity recognition, which consists of three parts: ALBERT pre-trained language model, bidirectional long short-term memory network (BiLSTM) and conditional random field (CRF). In the named entity recognition task, the model takes the text sequence as input, extracts feature representation through the pre-trained ALBERT model, performs sequence modeling through the BiLSTM network, and finally outputs the labeled sequence through the CRF model. The model shows good performance in the open source dataset NER task, and also achieves good results on the urban underground space disaster dataset, with a precision of 95.41%, a recall of 94.08%, and an F1 value of 94.74%. The experimental results show that the entity extraction effect of the model in the field of urban underground space disasters is better than some mainstream named entity recognition methods, and provides a valuable reference for the named entity recognition task in the urban underground space disaster neighborhood. In this experiment, only some event and literature data were collected for urban underground space disasters, which resulted in a single entity recognition task data, so it has certain limitations. In subsequent work, we will consider expanding the types of urban underground space disaster data and expanding the applicability of the ALBERT-BiLSTM-CRF model to assist in the construction of the urban underground space disaster knowledge graph. 1.2 Event causal relationship extraction method based on contextual time relationship

[0090] Relation extraction is a basic task in building a knowledge graph. Its purpose is to identify and extract specific semantic relationships between entities from text data. In natural language processing (NLP), entities usually refer to specific people, places, organizations, and time, while relationships describe the connection between these entities. The relationship extraction part mainly takes urban underground space disaster news events as the research object. In the above entity extraction experiment, entities including people, locations, events, time, and disasters have been extracted, that is, the extraction of time and space relationships in text data has been completed. In addition to these relationships, the event causal relationship accounts for a large proportion of the event content. Through relationship extraction, causal relationship triple data can be obtained. This paper divides the causes of urban underground space disaster events into four categories: natural factors, human factors, design and construction, and emergencies; and divides the results of urban underground space disaster events into four categories: casualties, property losses, social impacts, and environmental impacts. To address the above issues, this paper proposes an event causal relationship extraction method based on contextual temporal relationships, that is, a causal relationship extraction method that uses time entities to assist disaster events.

[0091] 1.2.1 Data Preprocessing

[0092] Relationship extraction is mainly based on news events. Since underground space disasters involve many types, this paper selects more typical flood disasters and pipeline disaster events as the main research objects. Flood disasters refer to water accumulation and flooding in urban underground spaces due to excessive rainfall, poor drainage systems or other factors. Pipeline disasters refer to disasters caused by ruptures, leaks, explosions and other accidents of various pipelines (such as water, gas, electricity and communications) in urban underground spaces.

[0093] The collected text data of urban underground space flood disaster events and pipeline accident disaster events were manually screened, and the text data was pre-processed with Python scripts for noise reduction. Finally, about 200,000 words of Chinese text data were retained, including about 150 comprehensive events (including several related events). In the word segmentation operation, this paper uses the LTP tool to segment words with the help of the entity dictionary mentioned above; in the annotation operation, this paper uses a semi-supervised learning method to annotate causal relationships, using a 1:1 ratio of manual annotation and automatic annotation by the LTP tool. The manually annotated data set is input into the LTP tool for learning, and then the LTP tool annotates the remaining 50% of the data set; for the causal relationship annotation of Chinese text data, the "BIO" (B-begin, I-inside, O-outside) annotation rule is used to annotate the more typical trigger words that represent causal relationships, such as because, due to, thus, caused, so, caused, triggered, caused and after..., which have obvious causal meanings, and the causal entities guided by each trigger word are annotated as "BX", "IX" or "O". Among them, "BC" (Begin-Cause) indicates that the character is located at the beginning of the cause segment; "IC" (Inside-Cause) indicates that the character is located at a non-starting position of the cause segment; "BR" (Begin-Result) indicates that the character is located at the beginning of the result segment; "IR" (Inside-Result) indicates that the character is located at a non-starting position of the result segment; "O" (Other) indicates that the element does not belong to any of the above types. The temporal entity annotation directly uses the results of the entity extraction experiment and is superimposed on the causal relationship annotation dataset. Figure 6 For marking rules, Figure 7 Specific examples are manually annotated.

[0094] 1.2.2 Causal Relationship Extraction Method Integrating Temporal Feature Matrix and Contextual Relationship

[0095] Disaster text data with events as the main body often use time as the index to describe the content of the event. Therefore, in the causal relationship extraction of urban underground space disaster events, adding time entities can improve the accuracy and reliability of causal relationship extraction. The effect of adding time features to the model on the extraction of causal relationships is as follows:

[0096] 1) Time sequence assists in determining causal relationships. By marking time entities, the time sequence of disaster events can be clarified, so as to more accurately determine the causal relationship between events. For example, if an earthquake occurs first and then a ground fissure appears, it can be determined that the earthquake is the cause of the ground fissure.

[0097] 2) The measurement of time intervals strengthens the causal relationship. By marking time entities, the time intervals between disaster events can be measured to further strengthen the causal relationship. For example, if a pipeline leak occurs immediately after an earthquake and there is a short time interval between the two, it can be concluded that the earthquake is the direct cause of the pipeline leak.

[0098] 3) Time dependency improves the inference of causal relationships. By integrating time entities, we can understand the time dependency between disaster events and then infer causal relationships. For example, underground flood disasters always occur after heavy rainfall, so we may think that heavy rainfall is one of the causes of flood disasters.

[0099] 4) Duration and frequency reveal causal relationships. Annotating time entities can provide information about the duration and frequency of disaster events, thereby revealing the causal relationship between events. For example, if heavy rainfall continues to occur and each time it causes the groundwater level to rise, it may be considered that the heavy rainfall is the cause of the water level rise.

[0100] 5) Support of time evidence. Time entities can also serve as time evidence to support the inference of causal relationships. In disaster events, time can be an important basis for proving causal relationships. For example, if an underground fire occurs at a certain point in time, and there is a report of a flammable gas pipeline leak before this point in time, it may be considered that the pipeline leak is one of the causes of the fire.

[0101] This paper adopts a causal relationship extraction method that integrates the time feature matrix. The operation is as follows: first, add the annotation of the time entity to the manual annotation operation of the training set, retain the time entity annotation set in the entity extraction process, and add the causal relationship annotation set; then, concatenate the encoding vector of the time entity in the text data with the word vector output by BERT, and add the word vector of the time entity to the original word vector output by BERT. This method has the following advantages: the fusion of the encoding vector of the time entity and the word vector of BERT can enhance the influence of time information at the model level, and adding the time entity to the manual annotation can increase the correlation between the time entity and the causal relationship at the data level. Specifically, by concatenating or adding the encoding vector of the time entity with the word vector of BERT, the model's perception of the temporal context can be enhanced, thereby better capturing the causal relationship between events. At the same time, adding time entities to the manually annotated data can help the model learn the association between time entities and causal relationships. When annotating data, time entities and causal relationships can be annotated at the sentence level or the fragment level so that the model can learn the importance of time entities to causal relationship extraction.

[0102] In summary, this paper proposes a BERT-BiLSTM-CRF causal relationship extraction method that integrates the temporal feature matrix. Figure 8 This is the model architecture diagram. Fig. 9 The following is the operation flow of the model. In the causal relationship extraction of urban underground space disaster events, BERT is used to encode words in the text sequence into semantically rich word vectors. Through pre-training, BERT can understand the contextual semantics of words, so as to better capture the complex semantic features in the event causal relationship, and add the time feature vector based on the word vector output by BERT, so as to better use the time entity to capture the causal relationship phrase; BiLSTM is used to further process the word vector extracted by BERT to obtain a more representative text representation. BiLSTM can transfer information in the sequence forward and backward in the time dimension to better understand the dependencies and structures in the text sequence; CRF is used to model the labels of the sequence output by BiLSTM. CRF can use contextual information to model the dependencies between adjacent labels, so as to more accurately identify the event causal relationship.

[0103] In simple terms, the word vector fused with the time feature matrix output by BERT provides rich contextual information based on time relations. BiLSTM further captures the local contextual features of sequence data based on BERT, and CRF combines global contextual information to predict labels for sequences and output results. This fusion model design can better handle complex event causal relationship extraction tasks and achieve better performance.

[0104] 1.2.3 Experimental results and evaluation

[0105] Under the premise of the same experimental environment, the model parameter settings are shown in Table 6. The data volume after causal relationship annotation is 2.2MB, which contains about 150 underground space disaster events. It is divided into training set, validation set and test set with a ratio of 8:1:1 and the experiment is carried out.

[0106] Table 6 Parameter settings

[0107]

[0108] Table 7 shows the experimental results of the causal relationship extraction model (BERT-T-BiLSTM-CRF) that integrates the temporal feature matrix and contextual relationship. The precision, recall and F1 value of the three labels Cause, Result and Other are listed in the table. As can be seen from Table 7, the recognition accuracy of cause phrases and result phrases is relatively high, up to 87.73%, and the recognition accuracy of sentences other than causal relationships can reach 95.49%.

[0109] Table 7 Results of causal relationship experiment

[0110]

[0111] In order to verify the advantages of this model, the BiLSTM-CRF model, which is currently the most mainstream model suitable for causal relationship extraction of Chinese text data, and the ALBERT-BiLSTM-CRF and BERT-BiLSTM-CRF models mentioned in the previous article are selected for comparative experiments. On the basis of the same parameter settings, the experimental results of the four models on urban underground space disaster data are shown in Table 8. It can be seen from Table 8 that: 1) Adding a pre-trained model to the method will generally improve the model accuracy; 2) The accuracy of the ALBERT-BiLSTM-CRF model is not as good as the BERT-BiLSTM-CRF model. Although the ALBERT model reduces the number of parameters by sharing parameters and may better capture the characteristics of the input data in this way, this feature can achieve a higher accuracy than the BERT model in some specific data sets; 3) The accuracy of the BERT-BiLSTM-CRF model that integrates time features is significantly higher than that of the BERT-BiLSTM-CRF model without adding time features, with an average precision of 89.46%, a recall rate of 88.32%, and an F1 value of 88.89%. The experimental results show that the model proposed in this paper improves the extraction effect of causal relationships of urban underground space disaster events by adding temporal relationships.

[0112] Table 8 Model comparison experimental results

[0113]

[0114] In summary, causal relationships are important types of relationships in text mining, and entities with causal relationships can be obtained through sequence labeling methods. Considering the connection between temporal relationships and causal relationships, this paper integrates the special matrix of temporal relationships on the basis of the BERT-BiLSTM-CRF network model to conduct experiments and accurately label the causal relationships. The experimental results show that adding the special matrix of temporal relationships can improve the extraction accuracy of causal relationships. The causal relationship extraction method proposed in this paper that integrates the temporal feature matrix and contextual relationships can effectively obtain the causal relationships contained in urban underground space disaster events.

[0115] 1.4.3 Summary

[0116] This paper mainly studies the knowledge extraction method of urban underground space disaster data and proposes two improved knowledge extraction models. First, the ALBERT-BiLSTM-CRF entity extraction model is proposed. On the basis of basic entities, time entities, underground space location entities, underground space disaster entities, and disaster loss entities are added. Good results have been achieved on the urban underground space disaster dataset, with a precision rate of 95.41%, a recall rate of 94.08%, and an F1 value of 94.74%, indicating that the model is superior to some commonly used named entity recognition methods and provides a valuable reference for the named entity recognition task of the relevant neighborhood of the urban underground space. Secondly, a causal relationship extraction method of the BERT-BiLSTM-CRF model with a fusion of time feature matrix is ​​proposed. The model adds time features on the basis of the BERT-BiLSTM-CRF neural network, which improves the accuracy of causal relationship recognition of urban underground space disaster events. Its average precision, recall rate and F1 value are 89.46%, 88.32% and 88.89% respectively, indicating that the model has certain advantages in causal relationship extraction of urban underground space disaster events.

[0117] An embodiment of the present invention also provides an electronic device. The electronic device includes: at least one processor, and a memory connected to the at least one processor in communication, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method described. Specifically, the above-mentioned memory and processor are connected together via a bus, and can be a general memory and processor, which is not specifically limited here. When the processor runs the computer program stored in the memory, it can perform the various operations and functions described in the embodiments of the present invention.

[0118] The embodiment of the present invention further provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented. The specific implementation can be found in the method embodiment, which will not be described in detail here.

[0119] Although the embodiments of the present invention are disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the embodiments of the present invention. For those familiar with the art, other modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scopes, the embodiments of the present invention are not limited to the specific details and the legends shown and described here.

Claims

1. A method for extracting event causal relationships based on contextual temporal relationships, characterized in that: include: Entity extraction is performed on Chinese text data of urban underground space disasters to obtain time entities, wherein the Chinese text data of urban underground space disasters selects text data of flood disasters and pipeline accident disaster events in urban underground space; Annotating the cause-effect relationship of the Chinese text data of the urban underground space disaster; The Chinese text data of disasters with causal relationship annotations were input into the BERT-BiLSTM-CRF model to extract the causal relationship of urban underground space disaster events. When the BERT layer in the BERT-BiLSTM-CRF model outputs a word vector, the encoding vector of the time entity is concatenated with the word vector output by the BERT layer to obtain a word vector containing a time relationship, and then the word vector containing the time relationship is input into the BiLSTM layer for processing; The word vector containing the time relationship then enters the BiLSTM layer, and the BiLSTM layer uses the time relationship to perform forward and reverse information processing on it, and then uses the CRF component to model the label of the text sequence output by the BiLSTM component; Before performing entity extraction on the Chinese text data of urban underground space disasters, the Chinese text data of urban underground space disasters is screened and pre-processed for noise reduction, the Chinese text data of disasters used for entity extraction is retained, and then word segmentation and labeling operations are performed. During the labeling operation, all time-related information is labeled as an entity type to obtain a time entity labeling data set; When the Chinese text data of the urban underground space disaster is annotated with a causal relationship, the time entity annotation data set is superimposed to obtain the disaster text data with the causal relationship annotation of the time entity added; The disaster text data with the causal relationship annotations of the time entity is input into the BERT-BiLSTM-CRF model to complete the extraction of the causal relationship of urban underground space disaster events.

2. The event causal relationship extraction method based on contextual time relationship as claimed in claim 1, characterized in that: The method for labeling the causal relationship of the Chinese text data of the urban underground space disaster includes: First, the Chinese text data of urban underground space disasters are screened and pre-processed for noise reduction, and the Chinese text data of disasters used for relationship extraction is retained. Then, the word segmentation operation is performed, and then the semi-supervised learning method is used to annotate the causal relationship. The manually annotated text data set is input into the LTP tool for learning, and then the LTP tool annotates the remaining data sets. The causal relationship annotation adopts the BIO annotation rule to annotate the trigger words representing the causal relationship, and annotate the causal entity guided by each trigger word as "BX", "IX" or "O", where "BC" indicates that the character is located at the starting part of the cause segment, "IC" indicates that the character is located at a non-starting position of the cause segment, "BR" indicates that the character is located at the starting part of the result segment, "IR" indicates that the character is located at a non-starting position of the result segment, and "O" indicates that the character does not belong to any of the above types.

3. The event causal relationship extraction method based on contextual time relationship as claimed in claim 1, characterized in that: When the disaster Chinese text data for entity extraction is annotated, the BIO part-of-speech tagging method is adopted, wherein "B" represents the start node of the entity, "I" represents all sequence labels after the start node, and the LTP tool is used for part-of-speech tagging, and the entity types are abstracted into five categories, namely "LOC", "PER", all time-related information, "ORG" and "DIS"; The disaster Chinese text data processed as above is input into the ALBERT-BiLSTM-CRF model to complete entity extraction.

4. A device for extracting event causal relationships based on contextual temporal relationships, characterized in that: include: Relationship extraction preprocessing module, which is used to annotate the causal relationship of Chinese text data on urban underground space disasters; A BERT-BiLSTM-CRF model is used to extract event causal relationships from input disaster Chinese text data that has been annotated with causal relationships. When the BERT layer in the BERT-BiLSTM-CRF model outputs a word vector, the encoding vector of the time entity of the urban underground space disaster Chinese text data is concatenated with the word vector output by the BERT layer to obtain a word vector containing a time relationship, and then the word vector containing the time relationship is input into the BiLSTM layer. An entity extraction preprocessing module is used to label the Chinese text data of urban underground space disasters according to entity type classification, where one entity type is all time-related information, and obtain a time entity labeling data set; When the relationship extraction preprocessing module performs causal relationship annotation on the urban underground space disaster Chinese text data, the time entity annotation data set is superimposed to obtain disaster text data with causal relationship annotation of the time entity added; The BERT-BiLSTM-CRF model is used to extract event causal relationships from the input disaster text data with causal relationship annotations of time entities added; In the BERT-BiLSTM-CRF model, the word vector containing the time relationship enters the BiLSTM layer again, and the BiLSTM layer uses the time relationship to perform forward and reverse information processing on it, and then uses the CRF component to model the label of the text sequence output by the BiLSTM component; The Chinese text data on urban underground space disasters selects text data on flood disasters and pipeline disaster events in urban underground spaces.

5. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the method according to any one of claims 1 to 3.

6. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images